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CHAPTER I 

SUMMARY OF THE MODEL CONCEPT 
AND ITS APPLICATION 

Millions of dollars are spent every day in the United States to 
improve the public schools, yet the quantity and quality of education 
available to many is believed inadequate to meet public demand. Since 
national human and physical resources potentially useable in the improve- 
ment of the schools are competed for by other national needs, only limited 
resources are available for schools. When improvements are desired 
and only limited amounts of the necessary resources are available, the 
efficiency of resource allocation becomes a critical problem. 

When there is not enough to do everything desirable, rational 
resource allocation decisions require information on the relative effective- 
ness of alternative types of expenditures. The goal is, of course, optimal 
resource allocation decisions, but optimization much more difficult than 
evaluation, since it involves the generation of programs, whereas evalua- 
tion does not. Optimization is a sensitive and complex problem, but if 
only evaluation can be carried out, this alone will be useful. 

Although millions are spent every day for the improvement of public 
schools, many more millions than are available would be needed to achieve 
all of the desired improvements. Thus, some means of estimating the 
expected returns from alternative types of investments in education is 
needed, so that the • most productive investments may be given priority. 

The relative efficiency or rate of return on investment of alternative 
education improvements may be measured by their relative cost-effective- 
ness, or effectiveness per cost expended. Both costs and effectiveness 
theoretically can be measured, but there are many practical difficulties. 

At least costs can roughly be estimated in advance on the basis of budget 
allocations. The prediction of effectiveness is much, more difficult. Yet 
without some such prediction, however crude or subjective, there is no 
rational basis for deciding between one education improvement program and 
another. 
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The prediction of education improvement program effectiveness 
can of course be based on recent experience with similar programs. 

Such experience, however, is not available for new programs, or pro- 
grams applied in new settings. Moreover, the effectiveness of previous 
programs is unfortunately only occasionally measured, recorded and 
disseminated. The experiences of teachers that are not measured and 

communicated to education policy makers are of no help in program 
selection. 

The large amount of data stored in every school concerning student 
achievement levels, student home addresses, and student destinations on 
eavmg school (dropout, graduation, higher education) do suggest that the 
effectiveness of education improvement programs can be measured in 
meaningful and concrete terms. If a program's impact on grades, 
(achievement), on the classes affected and number and quality of graduates 
and dropouts later on as a result of the program can be estimated, then 
there is already some measure of effectiveness. 

It would be desirable to measure the impact of educational improve- 
ment programs on the community as well as the student and the school 

Thus both community economic and social effects should be elements in 
any measure of effectiveness. 

H the population is broken down by achievement levels and these 
are matched with corresponding probabilities of unskilled, semi-skilled, 
and skilled labor jobs, business and clerical jobs, and professional and 
managerial prospects, then changes in expected lifetime earnings of 
students can be estimated on the basis of known averages in each earnings 
category. Thus the changes in students' prospective income categories 
as a result of an education improvement program may be estimated. The 

results also offer some indication of the economic impact on the community 
of programs for education impr'ovemonts. 

The social change component of any measure of the effectiveness of 
education improvements is more difficult to measure, because there are 
so many kinds of social changes, because many are inter-related, and 
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because they are difficult to measure. One major social issue toward * 
which much of the government' investment in education improvement- 
is directed is that of equality of educational opportunity. James Coleman 
has suggested that equality of educational opportunity is indicated by the 
absence of correlation between a student's socio-economic background 
and school achievement. If one can predict who will fail and drop out on 
the basis of race or family income, for example, then educational oppor- 
tunity is unequally distributed. 

Given this particular measure of social change - absence of 
correlation between socio-economic background and educational 
achievement - the social impact of an education improvement program 
may be measured by classifying the target student population into socio- 
economic or racial categories, and then determining if the predictability 
of educational achievement on this basis has declined as a result of the 
program. 

We have now discussed three types of educational effectiveness 
measures, student group-oriented, school-oriented and community- 
oriented. The student group-oriented measure is the academic achieve- 
ment change (grades) resulting from an education improvement program. 
The school-oriented measures are of three kinds: the consequences of 
these achievement changes in a given school population in terms of changes 
in the number dropping out of school, in the number selecting the various 
available courses of study, and the number and the quality (cumulative 
achievement level) of those graduating. The community- oriented measures 
are of two kinds: The average expected lifetime earnings potential of the 
population (economic change), and the reduced relation between socio- 
economic level and achievement indicating increased equality of educational 
opportunity (social change). 

The above measures of education improvement are available from 
school data in retrospect long after a program has been implemented. 
Unfortunately, such data are not available when the decisions must be 



made concerning which programs to implement. Hence, some means of 
predicting these measures of effectiveness of a program is needed 
before that program is implemented. Prediction always involves a theory 
about how one thing (the known, or "input") causes another (the predicted, 
or "output"). If a set of theoretical relationships intended to predict 
the unknown from the known is to be manipulated for a variety of cases, 
producing different "outputs" for each different "input" in accordance 
with some process simulating reality, this set of relationships is often 
called a model. 

The model is a simplified representation, or simulation, of those 
real world relationships between inputs (assumptions) and outputs 
(measures of consequences) believed most significant. The model may be 
expressed as a set of mathematical relationships between inputs and 
outputs, so that if the input is given the output may be determined. If 
a model of the educational process is given inputs of specific programs 
for a specific student population in a specific context, it should produce 
outputs indicating the changes in educational product. An education system 
cost-effectiveness model would show the changes in education costs and 
in measures of effectiveness resulting from changes in the education 
process from improvement programs. 

The education system cost-effectiveness model described in this 
report is intended to produce the measures of education effectiveness 
described above, in terms of student and school, achievement changes, 
and community, economic, and social changes. Each of these sets of 
predicted changes is associated with the estimated costs of the particular 
program causing those changes, so that each program has a set of cost- 
effectiveness output measures associated with it. Given a specific 
school and community setting, different educational improvement programs 
may be compared for their relative predicted cost-effectiveness, and the 
most cost-effective or efficient program may thus be rationally selected. 

The input data required by the model to produce these outputs must 
include quantitative descriptions of the school population to be affected 
by the improvement program, the improvement program itself, and the 
school and community settings in which the program and the target 
population interact. Unfortunately, it is an extremely complex task to 
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j describe school and community settings with sufficient quantitative 

accuracy to predict the results of the same program in different schools 
and communities, let alone the results of different programs in different 
schools in different communities. Pending the development of more 
quantitatively precise theoretical descriptions of how changes in com- 
munities interact with changes in schools and student achievements, it 
| seemed wise to limit this first attempt to develop an education cost- 

effectiveness model to a comparison of alternative programs within 

I 

| the same school and community settings. 

! In other words, the model described below is intended to predict 

t 

j the relative cost-effectiveness of alternative education improvement 

programs only within the same school- community setting, and not across 
! communities unless these communities can be assumed equivalent in all ‘ 

[, respects relevant to the education process. 

t 

It is also important to limit initial expectations for the model to 
| measures of the relative effectiveness of alternative programs. So long 

| as the relationships assumed to exist between inputs and outputs have 

not been empirically tested, corrected, and validated, these relationships 

j are unlikely to produce accurate absolute values of the various measures 

£ 

j of education effectiveness. For example, the expected change reduction in 

numbers of dropouts resulting from two alternative improvement programs 
might be 40% per thousand dollars for one program and 10% per thousand 
dollars for the other. This should be taken to mean only that the first 
program appears significantly more cost-effective than the second, not 
that it is exactly four times as cost-effective or efficient. 

The major problem in the design of any model, and certainly this 
one, is the determination of the quantitative relationships that translate 

s 

inputs (assumptions) into outputs (consequences). In the modeling of 
physical processes, these relationships are usually readily determined 
from empiracally verified theory, or "laws of nature". In the modeling 
of technological processes, such as a production line, the quantitative 
relationships between successive stages from input of raw materials to 

i 

output of finished goods are also well known, or can readily be determined 
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by observation and measurement. It is in the modeling of social processes 
involving human interactions that problems arise, because of the incom- 
plete knowledge concerning human decision rules and influence processes. 
Where these human decisions are numerous and repeated, as in macro- 
economic phenomena, statistics on their nature and distribution offer at 
least probabilistic data on relationships among variables. However, where 
the process being modeled involves modest numbers of individuals inter- 
acting in ways many of which have not yet been quantitatively measured, 
there are gaps in the linkage between causes and effects that must 
temporarily be bridged by hypothetical relationships remaining to be 
corrected or verified by subsequent testing of the model. 

In the model of concern here, there is a mixture of quantitative and 
qualitative knowledge concerning relationships among variables. The 
cost factors are largeley available in quantified terms. The measures of 
education effectiveness, however, depend on predictions of how students 
will respond to a variety of changes in school environment and the quantity 
and quality Gf instruction. These behavioral responses are the result 
of influence processes still only partially understood. The approach that 
has been taken in this model has been to quantify crudely some of what 

has been qualitatively and impressionistically described in the literature 
of education research. 

Some of the attitudinal variables believed decisive to the learning 
process, for example, may be given numerical index ratings roughly 
corresponding to the qualitative distinctions made in empirical research. 
The only presently available alternative would have been to omit these 
troublesome but significant variables, implying a spurious insignificance 
by such omission. The design preference has been for useful errors of 
commission rather than useless errors of omission. At least in this way 
useful areas of further research will be specified. 
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Note: 

The present cost-effectiveness model was intended to aid in 
decisions concerning alternative Title I programs. Title I is that part 
of the Elementary and Secondary Education Act of 1965 whose goal is 1 

to provide ‘'compensatory education" for the millions of school children 
deprived by poverty of the cultural environment found to be so important 
to successful schooling. Over one billion dollars are spent on Title I 
each year. 

The model is equally applicable to many alternative education 
improvement programs other than those in Title I. The effectiveness of 
i >,y education improvement program involving changes in school environ- 
ment, in the quantity or the quality of instruction, equipment, and facilities 
may be estimated by the techniques used in the present model. 
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CHAPTER II 



QUALITATIVE DESCRIPTION OF THE COMPLETE MODEL 

The OECE model has been developed for the purpose of assisting in 
the evaluation of alternative proposed Title I programs in any particular 
school district. It does not have the capability of allocating funds across 
communities, or of selecting the optimum mix of programs and describ- 
ing a precise menu of expenditures on various programs for a school 
district or districts. In the hands of a skilled user, it will help to 
determine the relative effects of any programs the user feeds into the 
model, and this somewhat limited capability can be a powerful tool for 

evaluation. 

► 

The OECE model is divided into several portions based on the 
chronological effects of Title I programs on the students undergoing the 
experience and on the input and output needs of the computer. There 
are four main portions of the model, and each main portion has from one 
to four submodels associated with it: 



FUNCTION 


SUBMODEL 




INPUT 


COST 


IMMEDIATE TITLE I 
EFFECTS 


INSTRUCTIONAL 

PROCESS 


LONGER- RANGE 
EFFECTS 


SCHOOL FLOW 

DROPOUT & TRUANCY 
CALCULATION 




COURSE OF STUDY 
SELECTION 




COMMUNITY EFFECTS 


OUTPUT 


EFFECTIVENESS 

OUTPUTS 
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The input part of the model is a straightforward data input and error 
checking procedure which serves to construct the data-base for the 
model. It takes as input punched cards with data describing the 
particular school or school district, the student population, and the 
community as a whole. After requesting user clarification of ambig- 
uous or incorrect information, the program will make up a data base 
tape for use with the actual simulation of the effects of the Title I 
program. It determines whether cost subtotals add up to give the total 
described on the input cards; in addition, it checks for numbers which 
seem to be unreasonable. For instance, if the computer program has 
been told to expect salaries to range between $ 2, 000 and $ 18, 000, 
and it is given information describing a school psychiatrist v/hose 
salary is $ 45, 000 for the school year, it will print out a note indicat- 
ing the inconsistency, and it will ask for confirmation or. correction 
of this salary by the user. 

The third function of the input portion of the model is to check 
for errors in the punching of data cards. If alphabetic information 
is detected in a location where numeric data is expected, the program 
will note it and inform the user. 

The immediate effects portion of the model describes the effects of 
the Title I program in the school during the time it is actually present. 
That is, if the proposed Title I program is one for remedial reading for 
fourth graders, the immediate effects portion of the model will calculate 
and describe the changes which the students undergo in the time period 
between the end of third grade and the beginning of fifth grade. It is 
at this point in time--during the ongoing Title I program- -that many 
programs peak in effectiveness, and large changes in studqnt attitude 
or achievement appear which may or may not damp out later. 

The longer-range effects portion of the model describes the progress 
of students after the Title I program is no longer in effect. In the 
longer.- range section, the results of a program may dairjp out (in terms of 
achievement) or they may grow and expand. The former condition seems 
to occur most often both in the literature and with the model. Changes in 




SEQUENCE OF SUBMODEL OPERATIONS 
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student achievement over the years, rate of dropout, selection of 
course of study by the students, and two community-oriented effects 
are calculated by this portion of the model. 

The output portion of the mod el takes the results of the simulation 
(both immediate effects and longer- range effects) and prints them out 
in a report to the user. The manner of organization and presentation 
of these results has been determined in conjunction with the Office of 
Education and will be in a format which is familiar to and understand- ■ 
able by educators, educational superintendents, and, in general, non- 
computer-programmer personnel. 

Below are described in somewhat more detail the subroutines 
which make up the functional elements described above. These sub- 
routines are explained in greater detail, each in its own chapter, 
later in this volume. 

The calculation of the immediate effects of a Title I program 
(those effects which occur while the program is present in the school) 
is carried out by the Instructional Process (IP) Submodel. This sub- 
model determines the change in student achievement level which takes 
place during each year that the Title I program is in effect. The IP 
calculates the extra achievement gain over that which would ordinarily 
occur in the absence of a Title I program, and gives as a final output, 
the total change in achievement for the appropriate year(s). 

The achievement change caused by Title I is computed on the basis 

m 

of two factors: the change in the overall effectiveness of the classroom 
environment, and the change in an overall student attitudinal variable 
which describes the students' resistance to learning based on their 
sociological backgrounds. As the effectiveness of the curriculum of the 
school increases, achievement will increase, as it will if the student im- 
pedance to learning variable decreases. 

The curriculum elements of Title I programs affect the classroom 
or curriculum portion of the Instructional Process submodel, and service 
elements affect the impedance portion. 

The overall curriculum variable reflects the values of indicators 
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wHch describe the quality of instruction and the quantity and intensity 
of instruction. By ascertaining changes (before and after Title I) in 
these indicators, the IP can determine the change in this variable. 

The student impedance variable is somewhat more controversial. 
Indicators of disadvantage are measured for the average child to determine 
the amount or level of impedance characterizing him before the intro- 
duction of Title I programs. The potential programs are then analyzed 
in terms of their relevance to these factors of disadvantage. That is, 
is there a service provided by the program which will tend to make up 
for the background disadvantage of the student? If so, then student 
impedance will decline, and achievement will improve. If there is no 
service offered by the proposed program, or the service provided is 
on the whole irrelevant to the needs of the child, then any change in 
achievement will have to come about through the curriculum half of the 
Instructional Process submodel. 

For example, a program which provides free lunches for children 
who have a history of low family income is deemed relevant since it 
tends to offset one of the disadvantages of these children. On the other 
hand, a program to provide eyeglasses for children who already see well 
on the average will not be relevant to any of their factors of disadvantage 
and will not decrease their impedance toward learning. From the 
literature^ it seems that many of the programs which are attempted and 
which are unsuccessful do not approach this important area of non- schoool- 
based student characteristics. 

Summarizing, the Instructional Process submodel predicts a change 
in achievement through the use of these variables, calculating change up 
to the point where the Title I program no longer is being operated for the 
particular target group in question. At this point, the longer-range effects 
portion of the model takes over. The first submodel in this part of the 
model is the School Flow submodel, which traces the achievement patterns 
of the students through the rest of their scholastic career up to the point 
where. they either drop out or graduate from high school. The School 
Flow submodel indicates the pattern of achievement for a group of students 
in any grade based on two factors: the achievement pattern for the group 
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in the grade immediately preceeding the grade in question, and a set 
of transition probabilities describing the likelihood of a student's 

moving from a particular pattern in the one grade to a particular pattern ' 

in the next. For instance, one of the probabilities might be described 
by saying, "If a student in this particular community or type of community 
passed English and math in grade 4 but failed science and social studies, 
the chance that he will pass English, math, and social studies but fail 
science in grade 5 is . 03. " 

If there is a Title I program in the third grade, the Instructional 
Process submodel will describe the achievement change for students 
in the third grade. Starting with the fourth grade, the School Flow 
model will extrapolate, from year to year, the achievement change for 
this group of students, keeping track of the whole group until its members 
either graduate or drop out. 

The Dropout/Truancy submodel is also in that part of the model 
which calculates the long-range effects of projected Title I programs. 

At the end of each school year this submodel determines the number of 
dropouts and the average truancy rate for the preceeding year. The 
truancy rate is recorded for output purposes, and the dropouts are re- 
moved from the group of students whose achievement average is to be 
extrapolated through the following years. The point at which students drop 
out is recorded and used both as an output and as data for the -calculation 
of potential lifetime earnings. 

Dropouts and truancy rates are determined by means of a relatively 
simple mathematical model, due to the limited availability of data 
describing their underlying causal factors. This complex problem could 
not be considered in as much detail as would have been desirable in a 
research model due to these data problems. The model developed here 
is a linear one, which multiplies the change in average achievement and i 

in the impedance variable described earlier for students by factors re- 
flecting the relative importance of these two variables in changing dropout 
rate in the particular school. The model will be programmed so as to be 

j 

adaptive, that is, given data on actual changes caused by Title I projects, it 
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will attempt to adjust itself to best possible values for these parameters. 

Another part of the long-range effects portion of the OECE model is 
the Course of Siudy Selection Submodel. This part of the model allows 
foi the choice (in those schools which offer a choice of courses of study to 
students) of a particular course of study such as, college preparation, 
vocational education, commercial/business, and so on. The results of 
this submodel are used both as output to the evaluators and as input 
data to the part of the model which determines the potential earnings of 
students. 

The procedure by which this submodel determines the change from 
historical patterns of course of study selection by the students as a result 
of Title I programs is covered in detail by Chapter VIII. In summary, 
the model receives changes in the mean values of achievement for 
students at the point in their school careers at which they select a course 
of study to follow. Based on the upward (or perhaps downward) shift in 
distribution of achievement for the students, the model makes more 
students eligible for those courses of study for which a higher achieve- 
ment mean is required. The course of study selected by the students 
as a result of the change in their achievement means will depend upon 
their backgrounds and the kinds of factors described in the Instructional 
Process Submodel. 

The outputs of the preceeding submodels are presented to the user 
for analysis, but they are also needed as data for the Community Effects 
Submodel. There are 1 two kinds of outputs to be derived from this sub- 
model. The first kind of output deals with the expected or potential 
lifetime earnings of the students, and the change in these figures as a 
result of the proposed Title I programs. The potential lifetime earnings 
figures are calculated on the basis of three important variables: 

1) whether he drops out of school or not, and if he does, when he does; 

2) the course of study he has selected in school, if the point of special- 
ization has been reached; and 3) for graduates, their achievement levels 
upon graduation. The second kind of Community Effects output is an 
indicator which describes Coleman's concept, "the equality of educational 
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opportunity. " That is, the association of student performance with 
student background is measured. To the degree that such an association 
is not present, to' that degree there is said to be equality of educational 
opportunity. 

The last portion of the overall model is that part which will provide 
output for study and analysis by the decision-makers involved in evaluation. 

The purpose of the model design is the development of a tool to aid decision- 
making, and to achieve this goal, the output from the model must be in 
a form which its users can understand and work with. The output shows 
the effects of a particular program, by itself, so that the general 
educational areas in which the program will have impact can be seen, 
and so that it can be determined whether the program will have beneficial 
or deleterious effects on the students in the school in question. More impor- 
tant, however, is the function of the program of evaluating alternative 
proposed projects in a particular school or school district so that they 
may be readily compared by Title I decision-makers. 

Following the next chapter, more detail may be found describing the 
subroutines of the model. 
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CHAPTER III 

QUALITATIVE ASPECTS OF THE MODEL DESIGN PROCESS 

The development of a model for evaluation of proposed Title I projects 
requires a through understanding of how and to what extent the various aspects 
of the overall student environment affect student attitudes and achievement. 
Complete understanding does not exist, and it is rarely possible to predict 
precisely what will happen to a group of students when certain changes are 
made in their school environment. On the other hand, it is not impossible 
to try to isolate those aspects of the environment which appear to have 
the most influence on student performance. Using the results of educational 
theory and experiment, the isolated variables can be related to one another 
in such a way as to simulate the actual educative process. 

The Office of Education Cost-Effectiveness (OECE) model is a first 
attempt at simulating the real world process of education in- a general 
framework. It was designed in response to the need for evaluating the 
relative effectiveness of alternative Title I project proposals for compen- 
satory education. Its principal function is to provide a more systematic 
and unbiased assessment procedure than either educated guesswork or 
straightforward historical comparison now provide. 

A model of an incompletely understood process is necessarily partly 
a simplification and a distortion. It will not be exhaustive or accurate 
because it does not comprehend every influence in the simulated process 
and because the relationships between even those influences that are 



known are not precisely understood. The OECE model is no exception to 
this rule. Its designers were faced with the problem of determining which 
of the known elements of compensatory education were of primary interest 
and which were not, in symbolically representing the process. The choices 
were conditioned by the ultimate purpose of the model and by the availability 
of supportive theory and required data. 

Underlying the model are important hypotheses. These hypotheses 
are derived from qualitative learning theory, some quantitative research 
results of Dave, BloonV and Coleman, . and assumptions made by the 
design staff at Abt Associates, Inc. , and their consultants. 
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Instruction is the principal production subprocess of education. 

The teacher, the curriculum materials, and the class itself represent 
the potential amount of learning that could be gained by any student in 
the room given satisfactory ability on the part of the student. The 
student may resist instruction. Were he to be completely responsive 
to his environment, the student would absorb that which could be 
learned in the classroom to the level of his ability. In effect, his 
resistance would be zero and the instruction transmitted to him would 
be completely converted into learning. 

The difference between What is taught and what is learned 
(ability aside) is the amount of resistance the student has to his 
environment. When a large number of children in a school are 
under-achievers, this achievement gap may probably be attributed 
almost entirely to resistance, not ability constraints. Whatever his 
reason- -lazines s, fatigue, dislike of the teacher, boredom, anxiety, 
or even lack of nourishment and proper clothing- -a student reduces 
the efficiency of teaching by his negative disposition. 

If grading in a given classroom is fair, then teaching efficency 
is the difference between what the teacher teaches and what the student 
learns, and will show up in the level of student achievement. That is 
to say, student achievement can be described in terms of the amount 
of potential instruction available and the amount of resistance the student 
poses against that instruction. The relationship between these three 
variablescan be expressed mathematically by calling achievement level 
the quotient of the amount of potential instruction divided by the student’s 
resistance to learning. 

This realtionship implies that achievement increases as the amount 
of potential instruction increases if the student's resistance either 
remains constant or decreases. It also suggests that if the amount 
of potential instruction were to remain constant and the resistance 
of the student were decreased, achievement would also rise. An 
analogy can be drawn between this simple relationship and Ohm's Law 
in electrical theory. Instruction flows from teachers to students, whose 



resistance lowers the resultant achievement force. 

How interdependent instruction and student resistance are poses 
an interesting problem. Can there be any achievement if resistance 
is abnormally high or instruction abnormally low? Common sense 
suggests that the two factors are conditional upon one another, that is, 
for certain ranges of each the other is operative. An experienced 
teacher of youngsters from culturally-deprived homes will no doubt 
support the contention that little learning takes place when students 
are negatively disposed toward schoolwork arid, receive no reinforce- 
ment outside the classroom. On the other hand, students from very 
privileged backgrounds are intolerant of low-level teaching and, 
presumably, increase their resistance as the quality of instruction 
decreases. 

This basic conceptual relationship between the level of instruction, 
the resistance of the student to learning, and student achievement can 
be translated into a model of compensatory education. First of all, by 
its very definition, compensatory education would seem to be addressed 
to students with resistances greater than zero. These students are 
underachieving because they are not learning all they are taught. 

Changes in the amount of instruction (within the proper range or 
resistance) will yield gains in student achievement. Changes in 
resistance will similarly increase achievement if the instructional 
level is adequate. 

Changes in the amount of instruction can be brought about by 
Title I projects aimed at improving the quality of teaching or the 
qualtity of the curriculum. Changes in resistance cannot be brought 
about directly by Title I. However, these changes in a student's 
disposition toward learning do take place as an indirect result of 
Title I service projects. The aim of service projects is to reduce the ill 
effects of improper health and welfare attention in the students' home 
environments. By so doing, service projects tend to increase students' 
receptivity to learning or, to put it another way, reduce their resistance 
to formal instruction. 
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The next step in defining the model is to identify those aspects of 
instruction, service, and student resistance which contribute most heavily 
to determining actual achievement and attitude change. Davy, Bloom, 

Coleman, Bernstein, and other social psychologists have made significant 
advances in identifying the characteristics of a student's environment which 
account for the large part of his achievement change. Unfortunately, the 
variables they have suggested in their research are not usually objectively 
measurable or easily accessible. The student's sense of mastery over his 
environment, his need-achievement, his parents' valuing of education, the 
norms of his peer group, the verbal facility of his teacher, and even the 
language patterns of his parents are identified as crucial influences on student 
achievement. Attitudes toward school appear to be less determinate than 
achievement and dependent upon even more inaccessible variables. 

To replace the important influences identified in the literature of 
educational research, indicators --reliable and accessible- -had to be found 
which suggested the most salient aspects of the crucial variables. The 
parents' level of education, for example, was selected as an indicator of 
the value placed by the parents on education. Other indices, like the recency 
of curriculum materials as an indicator of their interest and relevance to the 
students, were far less proximate because of the limitations on the standardiza 

tion and collection of objective data. 

These indices, representing the significant influences of the home 
and school environments on student attitude and achievement change, were 
then grouped into the four categories: achievement change, instruction, 
service, and student resistance to instruction. Each category received an 
overall index equal to a weighted linear combination of each of its variable 
components. The weights represent the influence attributed to each variable 
in its respective category. 

The basic relationships among the categories are operated by 
introducing a detailed description of a Title I project into the model. One 
cycle of the model will yield the attitude and achievement effects of a given 
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project at the time of impact. This information is then passed on to several 
other subroutines which extrapolate the forecast data out to grade twelve 
and into the community. The School Flow Submodel receives the information 
and computes the effect of an achievement change in one achievement category 
in the year of impact for all achievement categories through grade twelve. 

This model is based on the subject interdependencies of the core curriculum 
and computes changes in the probabilities of failure in all subjects and in 
all higher grades as a function of a change in one subject in any one grade. 
Changes in truancy rate, expected number of graduates and drop-out rate, 
and course of study selection are computed with the same attitude and 
achievement change information. Finally, the impact on the community is 
estimated along two dimensions, the increase in the expected average life 
earnings of the target population and the equality of educational opportunity 
in the community. 

Any given Title I project will have costs associated with its components 
of environmental change. When the effects are tabulated for these changes, 
the individual project effects and costs are measures of the relative cost- 
effectiveness of the project. 

The first step in the design of any model is the definition of outputs 
to serve the model's objectives. The outputs of the process of compensatory 
education are defined as changes in student attitudes and achievement. 
Presumably, these changes will vary in accordance with the differential 
emphases of various Title I projects and the amount of effort expended. 

For purposes of evaluation, however, the effects of Title I projects have 
to be combined with their costs in order to arrive at a measure of 
relative effectiveness. Two competing projects, for example, may yield 
equivalent achievement gains for the target population with widely different 

m 

costs. The cheaper cf the two projects would be the more cost-effective. 

Two other projects may yield equivalent achievement gains in incommensurate 
categories, such as reading and arithmetic, at the same cost. In this 
case, cost-effectiveness is identical for the two projects, unless either reading 
or arithmetic is considered more beneficial to the students affected by the 
project. 



Following the designation of outputs is the selection of instrumental 
variables and data inputs. The instrumental variables in the compensatory- 
education process are those influences in the school which contribute most 
to student attitude and achievement change and which can be affected by- 
projects under Title I. Since Title I is divided into two categories, personal 
services and instruction, these same categories were used in classifying school 
environment variables. The logic employed in the classification was straight- 
forward. Both categories were divided into measures of their quality and 
quantity. The combination of these two measures represents the total impact 
service and instruction have on the ultimate changes in student attitudes 
and achievement. 

Data inputs consisted of a detailed description of the proposed Title I 
project and a characterization of the target population. The Title I project 
was described in terms of its costs and the changes it purported to make in 
the quality and quantity of services and instruction provided by the school. 

The components of Title I changes are described in terms similar to those 
of the instrumental variables so that the model does not go through an 
unnecessary process conversion. 

Students in the model are characterized in two complementary ways. 

The first description is an ethnic/income breakdown. The model deals with 
four so-called student- types: whites whose parents' income exceeds $2,000, 
non-whites whose parents' income exceeds $2,000, whites whose parents' 
income is less than $2, 000, and non-whites whose parents' income is less 
than $2,000. This breakdown exists so that possible differences in student 
background and resulting impedance to learning may be rated. The second 
dimension is so-called student impedance. Impedance represents the degree 
of scholastic disadvantage that characterizes each, student type. It is a 

* 

combination of home and school background factors which are presumed to 
retard learning in the target groups. 

In a model, input variables are combined with instrumental variables 
which interact with one another to product the outputs. The combinations 
and interactions of the model variables require a set of decision rules and 
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precise designations of the relationships among the variables. These rules 

and relationships constitute the theory of the model. 

The theory of the OECE model is relatively simple. It consists of 
two basic relationships and a number of assumptions. The first relationship 
is that the decrease in student impedance is proportional to the total increase 
in the quality and quantity of personal services provided by the school. This 
relationship assumes that improved services in the school will tend to 
reduce the scholastic disadvantages accumulated by the target students in 
their homes and in previous school years. The change in scholastic dis- 
advantage forecast by the model is taken to be equivalent to the change in 

student attitudes and is output as such. 

The second relationship is that the change in student achievement is 

directly proportional to the total change in the quality and quantity of 
instruction and inversely proportional to the total change in impedance. 

This implies that achievement change can be accomplished by holding 
impedance constant and increasing instruction, by holding instruction 

constant and decreasing impedance at the same time. 

Certain rules, however, govern the behavior of these relationships. 
Service components of Title I projects are matched against the particular 
disadvantages of the target population before any impedance change is computed. 
If the service improvements are not relevant to the student disadvantages, 
then no impedance change is recorded. A second constraint on the behaviors 
of the variable relations is the imposition of thresholds. One example of the 
operation of this constraint occurs also in the computation of impedance 
change. Because impedance actually represented student attitudes, there 
was much evidence to suggest that there was a practical limit on the amount 
of change that could occur in a single year irrespective of'the amount of 
service improvement in the school. .Thus an upper limit was placed on the 

impedance change relationship for any given year. 

These two relationships indicate what immediate impact a Title I 
project will have on student attitude and achievement. Evaluators interested 
in the longer-range effects of a given project can turn to the following four 
features of the model: the effect of a change in achievement in the year of 
impact on achievement in future years all the way to grade twelve (School 
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Flow Submodel); the effect of changes in achievement and impedance in 
the year of impact on student absence (Truancy Subroutine) and drop-out 
proneness (Drop-out Subroutine) to grade twelve; the effect of changes in 
achievement and impedance in the year of impact on student selection of 
a course of study (Course of Study Selection Subroutine) and expected lifetime 
earnings (Community Effects Submodel); and, finally, the effect of changes in 
achievement in the year of impact on the equality of educational opportunity 
(Community Effects Submodel). 

The School Flow Submodel has been developed with the basic assumption that 
early failures in academic subjects lead to later failures in other subjects. 

This effect is likely to spread in later years of school, due to the increasing 
reliance of new subjects on those previously taught. For instance, reading 
ability is necessary for most subjects from early elementary school on, 
mathematical skills are necessary for a wide variety of subjects later on 
in school; science and social studies courses often build on previous 
courses. Detailed study of achievement data collected by Abt Associates 
staff members both in the greater Boston area and in Iowa revealed that 
patterns of spreading achievement failure do indeed exist both for achievement 
test data and for teacher-assigned grades. 

The subject-grade interdependencies are simulated in the model by 

the use of a one-stage Markov model; probabilities of passing a set of course 

in one grade are determined by the pattern of courses passed by the student 

in the previous grade. Tests of the Markov property of some of the data 
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gathered have been encouraging and are described in more detail in the 
Notes at the end of Chapter VI. 

Drop-out and truancy phenomena were studied in some detail by 
Abt Associates staff members and a relatively complex model was developed 
(described in the September and February interim reports). Unfortunately, 
due to two factors, the model had to be simplified. These factors were: j 

. (1) the absence of data on many of the variables which were felt to be important, 
and (2) the insensitivity of many of the variables to changes attributable to 

Title I programs. The original model dealt with such influences as the home j 
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environment, peer group pressures, community income distribution, 
employment rates, and family size and crowding. In its simplified version, 
the model consists of a simple linear relationship of changes in the dropout 
and truancy rates to changes in achievement and attitude. This simplification 
is assumed valid for small changes in the rates. 

The Course of Study Selection Submodel is a straightforward 
mathematical development of the assumption that for students of a given 
background, selection of courses of study is a function of student achievement; 
the patterns of student choice are based on historical achievement data, and 
shifts in achievement will result in shifts in choice of course of study. 

The Potential Lifetime Earnings portion of the Community Effects 
Submodel is based upon research into studies of the association of lifetime 
earnings with educational level achieved, and census report data. As 
grade of school-leaving increases, as achievement at graduation increases 
(generally), and depending upon occupation chosen (and before that, course 
of study selected), lifetime earnings increase. 

The Index of the Equality of Educational Opportunity is based on 
Coleman's concept of the relationship between school achievement and socio- 
economic background and is described in detail in Chapter IX. 

A Cost Subroutine outputs to the user of the model the total costs 
of the individual projects. When these costs are weighed against the immediate 
and long-range effects of projects, the relative cost-effectiveness of any one 
project in a set can be computed. Incommensurate effects that are equally 
cost-effective can be resolved by assigning priority values or benefits to 
the individual categories of effects. 

The OECE model was developed for the purpose of assisting in the 
evaluation of alternative proposed Title I programs in any particular school 
district. It does not have the capability of allocating funds across communites, 
or of selecting the optimum mix of programs and describing a precise menu 
of expenditures on various programs for a school district or districts. In 
the hands of a skilled user, it will help to determine the relative effects of 
any programs the user feeds into the model, and this somewhat limited 
capability can be a powerful tool for evaluation. To build a model which 
could generate and optimize programs for a group of school districts 
would require a great deal of additional resources and effort. 
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...A Brief Profile of the OECE Model 



The Model Will: 


Function 


The Model Won't: 


Deal with groups of students 

Deal with students below 
national norms 

Indicate changes in student 
group achievement 


Students and student 
change 


Deal with individuals 

Evaluate programs to 
raise achievement of 
students above national 
norms 


Indicate changes in rate 
and year of dropouts 

Indicate increased or 
decreased numbers of 
high school graduates 

Indicate changes in course 
of study selection where 
applic able 


School change 


Simulate changes in the 
administrative policy 
in a school district 


Indicate changes in potential 
life-time earnings 

« 1 

• 

Indicate’ changes in equality 
, of educational opportunity 
• 


Community change 


Simulate change in the 
home as a result of 
Title I 


Compare the cost-effective- 
ness of proposed Title I 
and other educational im- 
provement projects within 
a school district 

Aid decision-makers 

Determine relative effective- 
ness of proposed Title I 
programs 


Evaluation 


Compare proposed Title I 
projects across districts 

Make decisions 

Determine absolute 
effectiveness of proposed 
Title I programs 


Need commonly available 
data 


Data input 


Give results more precise 
than the input data 
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CHAPTER IV 



THE COST /INPUT SUBMODEL 

The Cost/input Submodel converts the data concerning a 
school system and its proposed Title I projects into a form 
acceptable to the main model, checks the data for certain detectable 
errors, and generates reports that facilitate the comparative 
evaluations of proposed projects and the correction and verification 
of the data about those projects. 

The processor consists of two elements: the pre-processor 
and the data base preparation program. The first of these elements 
checks errors and generates reports. The second element arranges 
the data into the sequence required by the main model's, logic. 

These two elements are operationally independent of each other. 

The pre-processor is the first element to receive the data for a 
particular school system. It examines them for errors in complete- 
ness or consistency and produces, if necessary, the appropriate 
error indications. In addition to the messages, it generates listings 
useful for cross project cost comparisons and a listing for verifying 
the schedule of target group-project combinations to be simulated. 
These various reports go to whoever is running the model and indicate 
whether the data should be corrected and resubmitted to the pre- 
processor or left as < they are and submitted directly to the data base 
preparation element. * 

The second element takes the presumably error- free data and 
combines it with various data that are independent of the particular 
school systems or projects being evaluated to produce a 'data base on 
an input tape for the main model. Figure 1 shows the overall process. 

Before examining the operation of these elements in more detail, 
some consideration should be given to how the nature of the model's 
data needs have shaped the processor. As noted in Chapter V, the 
"target group" concept is central to the simulation's structure. A 
simplified description of the model is a device that predicts the 
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relative changes in the performances of target group members on 
the basis of expected increases or decreases in measures of 
selected factors of the school environment of the target group. 
Necessarily, then, data about proposed projects must be oriented 
about the target group concept to be useful to the simulation. In 
general, however, the data currently gathered for the evaluations of 
proposed Title I projects are not oriented about target groups as 
they are defined for the OECE model. And, indeed, those responsible 
for planning projects at the local level may very well structure their 
programs so as to include student sub-populations that differ 



considerably from the particular target group configurations assumed 
here. A result of these differing approaches is that a project seen 
as an entity by its local planners is seen by the model as a set of 
projects affecting a number of different target groups differentially. 

When this situation exists, and if the model is to assist in the 
evaluation of this set, the local administrator must supply data 
disaggregated to ard oriented about this target group project level. 
Unfortunately this disagregation cannot always be done meaningfully 
by simply prorating the aggregated values on a populatio basis. 
Instead--as may be seen from an examination of the questionnaire-- 
the administrator must supply somewhat detailed breakdowns of the 
data by target. group and subject areas. 

The first and most obvious result of these target group consider- 
ations is the form assumed by the cost reports generated by the 
pre-processor. Rather than being concerned with an entire local 
project, they focus on the costs for a target group and the costs per 
target group member. These costs are broken down in a number of 
standard categories (see OE Form 4305, Items 8a and 8b) and into 
the specific factors which are actually incorporated into the model. 

An example of a cost report is shown in Figures 2A and 2B. 

Besides generating this cost report, the pre-processor attempts 
to ensure that the input data is valid. This data is of three different 
types; two of these types may be inferred from the discussion above, 
specifically, data must be included that is descriptive of the sub- 
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COST REPORT 



PROJECT NAME: Intensive Teaching 
PROJECT ID NO. : 4 



Ferndale, 

Massachusetts 



Grade:3 



Target Population ID No. 1020 
Distribution by Student Type 
White More Than $2000 Family Income 
White Less Than $2000 Family Income 
Non- White More Than $2000 Family Income 
Non- White Less Than $2000 Family Income 
Total Population 



Project Budget for Target Group 1020 



Cost For 
Entire Tar 
get Group 



Administration 
Instruction 
Attendance Services 
Health Services 
Pupil Transportation Services 
Operation of Plant 
Maintenance of-Plant 
Fixed Chargers 
Food Services 
Student Body Activities 
Community Services 
Remodeling (Less than $2000) 

Equipment 

Professional Services for Sites 
Sites and Site Additions 
Improvements to Sites 
Professional Services for Building 
Remodeling ($2000 or more) 

Equipment (obtained as part of construction) 0 

Input Total 8000 

Computed Total 8000 



0 

8000 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 



Figure 2A 
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200 
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COST REPORT 

PROJECT NAME: Intensive Teaching 
PROJECT ID NO. : 4 



Ferndale, 

Massachusetts 




The following increases (or decreases) are expected for target group 1020. 



Service Factors 

Change in Number of Service 

Professionals 0 

Change in Service Exposure Time 

Hours/Day 0 

Days/Week 0 

Weeks/Year 0 

Cost of Service Supplies and Equipment 0 

Instruction Factors 

Change in Number of Teachers 2 

Change in Salary Per Teacher 0 

Change in Student Exposure Time 

Hours /Day 0 

Days/Week 0 

Weeks/Year 0 

Cost of Instructional Supplies 0 

Change in Number of Desks 0 

Change in Number of T exts 0 

i 

Change in Text Copyright Date 0 



Figure 2B 
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projects and of the target groups. In addition, data must be included 
about some aspects of the entire school system for which the proposed 
project is being evaluated. For each type of data set there is a 
group of tests in the pre-processor which examines the relationships 
among the data within that particular set. Another group of tests 
is responsible for exainining the relationships between data sets. 

Figure 3 indicates the overall way in which these groups of tests 
and the cost report generation are related to each other in the pre- 
processor. Some of the details of these tests are described below. 

Parts of the school system, proposed project, and target group 
data sets are checked in a similar manner. Answers to certain sections - 
of the questionnaire provide data at the individual item, "subtotal, " 
and "total" levels of aggregation (c. f. sections A. 1, A. 2, A. 3, B. 6, 

C. 4, & C. 5 of the questionnaire, Volume II, Chapter III). Independent ' 
computation by the pre-processor of the totals using the given item and 
subtotal information provides a simple and effective test of the 
arithmetic correctness of these data. Detection of disagreements 
among these data causes printing of the appropriate message from 
Table, 1 . 

The data describing the proposed projects are checked to see if 
the project’s budgeted costs (such as those indicated in Section B. 6 of 
the questionnaire) imply the proposed changes in the school environ- 
ment (such as those indicated in Sections B. 4 to B. 5) and vice versa. 
Table 2 indicates the tests for these implications and the messages 
generated if there are seemingly false implications. 

Besides the above intra-set tests, there are tests which check 
inter- set relationships. For any one school system there are likely 
to be several different combinations of target groups and proposed 
projects. For. example, a school system may wish to evaluate the 
cases where Project X is applied in the 3rd grade, where Project X 
is applied in the 5th grade, where Project Y is applied in the 3d 
and 4th grades simultaneously, and where Project Z is applied in the 
5th grade. If simulations are to run correctly, it is necessary that 
the characteristics of the affected 3d, . 4th, and 5th grade students 
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TABLE 1 

INTRA-DATA-SET ERROR MESSAGES 




Data about the student population in section A. 1 for grade i 
are inconsistent 




* 






n 

0 



Data about the number of truants in section A. 2 for grade i 
are inconsistent 

Data about the number of dropouts in section A. 3 for grade i 
are inconsistent 

The student population in the grade levels which have courses 

of study as indicated by section A. 5 does not agree with the 
population as indicated in section A. 1. An error is indicated 
in either A. 5 or A. 1 or A. 4 

Row j in section C.4 or C. 5 is inconsistent 

The itemized and total costs in section B. 6 are not inconsistent 
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TABLE II 



PROJECT COST CONSISTENCY REPORT MESSAGES 



1. There is an increase in administrative costs with no increases 
in any other areas. 

2. Student hours increase with no increase in operation budget. 

3. The square footage per child increases with neither an increase 
in construction budget nor a decrease in population. 

4. There is a budget for construction; with population steady or 
declining square footage per child does not increase. 

5. There is a budget for professional sites services with no 
budget for other site related activities . 

6. There is a budget for site acquisition or improvement with no 
budget for site services. 

7. There is a budget for professional building services with no 
budget for building or major remodeling. 

8. There is a budget for building or major remodeling with no 
budget for progessional building services. 

9. Texts are updated with no entry for instructional supplies and 
equipment. 

10. The project is described as Type I but is no budget for health and 
food services. 

% 

11. The project is described as either Type 2 or 5, but there are no 
increases in instruction, equipment, or the instructional environ- 
ment factors. 

12. The project is described as Type 3 but there are no service 

environment, pupil transportation, or student body activities 
changes. 4 '• 

i r • • - 

13. The project is described as Type 4, but there is no health 
service budget entry or vice versa. 
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TABLE III 



INTER -DATA- SET ERROR MESSAGES 



. * 

1. Data about a grade J target group has been given. No 
a project aimed at that grade has been given. 



2 .. 



Data about a project aimed at grade jhas been given, 
about a target group in that grade has been given. 



data about 



No data 



and of Pi oj ects X, Y, and Z be present and that there be agreement 
between the project and target group data sets as to which projects 
are aimed at which students. Table 3 shows the messages and 
tests that are made to determine if such ensembles of target groups 
and projects have been correctly related. Regardless of whether 
inter- set tests do or do not find legal relations, a report such as 
that shown in Figure 4 is generated showing how the simulation 
runs for a particular school system will be scheduled. 

Once the pre-processor has run and has produced the cost, 
sequencing, and error reports, these reports are examined by the 
person responsible for the model run to determine whether corrections 
should be made in the data. If corrections arc needed, then after 
they are made the data should be resubmitted to the pre-processor 
This cycle continues until an error-free deck is obtained. Once 

such a deck is obtained, the data go.. to the data base preparation 
section. 

The data base preparation program is responsible for 
arranging the error- free data into a form acceptable and meaningful 
to the simulation. This involves two processes. The first of these is 
ari anging the given data so that the logical requirements of the main 
model are satisfied; the second is adding those data whose values 
are either independent of school system or are applicable to most 
schools systems whose population are of a certain demographic type. 
Examples of such data are the probability matrices for the School 
Flow Submodel and the values for the various coefficients in the 
Instructional Process Submodel equations. These latter data are 
supplied to the data base preparation program from thejmaster 
coefficient file which will have been previously compiled on the basis 
of research and expert judgment. Figure 5 indicates the pattern in 
which the various types of data are put for the main model’s use. 

Emphasis in the present Cost/Input Submodel design effort has 
been placed on providing a framework for providing cost and other data 
in a usable form to the user and to the instructional process, school 
flow, community, and the dropout/truancy submodels. It is felt that 
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j SIMULATION SEQUENCE REPORT 

I SCHOOL SYSTEM: Ferndale, Mass. 

; The following combinations of projects and target groups will be 

[ evaluated: 

is 

! 



Target Group 
Grade 


Target Group 
I. D. 


Project Name 


Project I. D. 


3 


1000 


Read Readiness 


1 


3 


1000 


Field Trips 


2 


4 


1001 


Field T rips 


2 


5 


1002 


Rem Reading 


3 




i 



Figure 4 
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INPUT SEQUENCE TO THE MAIN MODEL 



Mark Indicating School 
System Beginning 



Master File Supplied Data 




General School 
System Data 




Number of Target Groups 




1 st 

Target Group Data 




Number of Projects To 
Be Evaluated For 1st 
Target Group 




1 st 

Project Data 


0 O 0 


M th 

Project Data 


O O O 
o o o 
o o o 


N th — 

Target Group Data 




Number of Projects 
To Be Evaluated for N th Target 
Grouo 




1st 

Project Data 


0 0 0 


Mth 

Project Data 



Figure 5 
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the actual operation and refinement of these models should occur 
before the development of more refined methods of assigning costs. 

As these models are tested, the desirability and utility of including 
present discounted value, typical particular item and package costs, 
projected support needs, and multiple year cost considerations- - 
all of which involve considerable data reduction problems- -will arise. 
The design described here serves as a starting point for a submodel 
that reflects how costs are actually used in the model and that 
provides a data input interface between the model and its users. 

Since the model serves this interfacing function, its further 
development depends on that of the model. 
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CHAPTER Y 



THE INSTRUCTIONAL PROCESS SUBMODEL 



Central to the functioning of the overall cost-effectiveness model 

is the procedure for computing the immediate impact of a Title I project 

on the attitudes and achievements of its target population. This procedure 

is called the Instructional Process Submodel. It simulates the effects of 

» 

changes in instruction and school services on student academic achievement, 
allowing two or more projects competing in a school district to be compared 
for their relative effectiveness. 

Three assumptions underlie the Instructional Process Submodel. 

The first is that under achievement and lack of motivation among students 
from low-income homes is environmental rather than hereditary. The 
second assumption, following from the first, is that proper changes in the 
school environment- -more personal services and better instruction- -can 
contribute significantly to reducing learning difficulties and eventually im- 
proving student attitudes and performance. A more ambitious future 
model would also have to consider the impact of home and community 
changes on student achievement, but this is beyond the scope of the present 
effort. 

Since the Instructional Process Submodel deals - only with changes in 
the school environment and not with those of the home or community, its 
measures of effectiveness are relative rather than absolute. Thus, the 
third assumption of the submodel is that among Title I projects being 
compared, the home and community environments are and remain the 
same. The extent to which the ceteris paribus assumption does not 
hold is the extent to which effectiveness is .measured relatively. 

The Instructional Process Submodel changes achievement in 
response to Title I changes in two factors: (1) the overall effectiveness 

of instruction in the classroom, and (2) the resistance of the students to 
•learning. The predicted achievement change is positively related to 

t . 

instructional quality and inversely related to student impedance to learning. 
In simplified form: 
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where 




A A = change in achievement 
C = change in instructional 
effective nes s 

.1 

L\ Z = change in student impedance 
to learning 




Title I programs are of two basic kinds, service -oriented and instruction 
oriented. The service programs affect the student impedance term by 

providing the student with medical and welfare attention. • The instructional 
programs affect the instructional term through the improvement of the 
quantity and/or quality of instruction. If instructional effectiveness is 



increased or student impedance to learning is decreased, achievement 
will be improved. 



Impedance to learning is determined for students as a function of six 
factors which are closely linked with under achievement. These factors 
are: . - 

parental income level 
parental education level 
family solidarity 
student handicap 
student achievement level 
peer achievement level 

If a group of students have parents whose incoirie level is very low and 
have had only an elementary education and if the students are far behind 
national norms in achievement and their peers are also far behind these 
norms, and if their fathers have deserted and their mothers work, and if 
the students all have some sort of physical handicap such as nearsightedness, 
then the value computed by the model for student' impedance to learning would 
h© near the maximum. 

The effectiveness of Instruction term if divided into the quantity 
and quality of instruction. Quantity of instruction is measured by class- 
room exposure time, which combines duration and intensity of the index 
of instructional intensity and includes the teacher /student rat , text/ student 
ratio, desks /student ratio, and the materials expenditure /student. Quality 
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of instruction is an extremely difficult factor to isolate with data that 
is widely available data. It is calculated in the model by indicators 
such as the recency of curriculum materials and teacher effectiveness 
measures such as age, experience, and number of degrees received. 

These, in brief, are the indicators used in calculating the change 
in achievement. These factors, being of primary important, are now 
covered in greater detail. 

Many aspects of a student's environment affect his attitude toward 
school and his performance in the classroom. The variables that appear 
to be critical to the Instructional Process Submodel's outputs are the 
student's previous performance, his family background, the influence 
of his peers, the quality of the classroom and teachers, the interest of the 
curriculum, and the personal services provided by the school. For the 
model to operate, each of the considerations has to be described in such 
a way that all can be combined together and . : 

by the change components of a Title I project. Since there exists no 
standardized measure for any one of these environmental influences, the 
solution to this problem is to find appropriate numerical indices for each 
of the enviionmental characteristics. 

One problem in the assign^ment of values to indices is that there 
is often wide disagreement among educational experts as to the value of 
particular teaching methods or curriculum materials. 

The availability of data is another constraint on development of 
numerical indices in the model. Some indices not used might have been 
preferable to others used, but access to data through school records 
is limited. As a result, the final indices are the most accessible 
approximations to the environmental determinants of student achievement 
change. 

These indices describe three groupings of variables: the student, 
the classroom, and the school. Title I projects are described in terms 
of the classroom and school indices. Each grouping consists of a set of 
indicators which are either influenced directly by Title I, as in the case 



of classroom and school categories, or are affected by Title I through the 



operation of the model, namely the student category. Below are the actual 



indicators used in each category: 



Student 



Classroom 



School 



parents' income 
parents' education level 
physical handicap 
family solidarity 
classroom peer under - 
achievement level 
own under achievement 
in basic skills 



/^recency of curriculum 
materials 

/^teacher experience, 
v age, number of degrees 
Lite ache r / student 
Lj.ext /student 
/ldesk/stude-nt 
/\$ material aids /student 
classroom exposure 
time 




A P re sence of service 
j\^ student exposure to 
service 

/\space/ student 

^paraprofessionals / 
student 

service materials/ 
student 



These indicators represent variables in the environment which are presumed 
to account for most of the variance in student attitude and achievement change. 
In the student category, it should be noted that though some of these indicators 
appear to be fixed with respect to Title I, they combine to represent attitudes 
which can in fact be affected by Title I. This attitude of negative disposition 
toward learning is referred to as impedance to suggest that a student's 
resistance to instruction is analogous to a circuit's impedance to current 
in electric circuit theory. 

In the Instructional Process Submodel, classroom, school, and student 
indicators interact to give estimates of change in impedance and achievement 
levels. The assumption is that a student's sociological background, 
his performance record, his scholastic environment, and his future attitudes 
and achievements are intimately related, Thisis simply a restatement of two 
of the assumptions upon which the entire model is built. 

These indicators alone are not sufficient to generate attitude and 
achievement changes. Some of the indices are more significant than others, 
and some become significant only when certain conditions obtain. How and 
to what extent all of these indicators relate to one another is the meat of the 
model. Exact quantitative relationships among the individual indicators 
do not exist in present educational theory. The model therefore will use 

expert judgments as to how much impedance decreases when Title I changes 

{ 

certain aspects of the environment and how much achievement increases 

♦ 

in the face of decreasing impedance and increasing instructional effectiveness. 
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Changes in instructional effectiveness come about through the use 
of the instructional portion of the submodel. Title I projects are directed 
primarily at the classroom and are designed to improve the quality or to 
increase the quantity of the instruction the target students are receiving. 

The indicators listed under "Classroom” above are assumed by the model 
to account for most of the variance in achievement change that can be 
attributed to changes in instruction. 

These indicators represent the significant influences the teacher, 
curriculum, and classroom are presumed to have on the change in student 
achievement. In the Instructional Process Submodel, they complement 
the influences of the personal services provided by the school. The 
indicators fall into two categories: those representing the quality of instruc- 
tion, and those representing the quantity of instruction. The quality of 
instruction describes the interest level of the curriculum materials and 
the effectiveness of the teacher. Since neither of these lends itself 
immediately to objective or standardized measurement, indicators had to 
be chosen which would suggest the most salient aspects of curriculum 
and teacher quality. 

In the case of the curriculum , it was felt that the "recency of materials' 
provided a fairly reliable indication of how interesting to a group of students 
a given curriculum was likely to be. Teacher effectiveness is measured 
in the submodel along three dimensions: age, experience, and number of 
advanced degrees. Research conducted by members of the Abt Associates 
staff revealed that teacher effectiveness correlated with no single charac- 
teristic of the teacher. A limited relationship was found for number of 
degrees received and experience. It also appeared to be true that very 
young and very old teachers generally performed less effectively than 
middle-aged teachers. The most desirable index would -have been a measure 
of teacher aptitude or verbal facility, both of which have been found to be 
better indicators of teacher qualtiy. Unfortunately, neither type of data is 
commonly accessible. As a result, teacher quality is judged in the sub- 
model as a combined index of age, experience, and degrees received. 

The effectiveness of instruction is defined in the submodel not only 
by the quality of the teaching and the materials but also by the intensity and 
duratio n of contact between the student and the agents of his learning. 
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Quality of instruction, then, is complemented by the quantity of instruction 
that passes between teacher, curriculum, and student. Four numerical 
ratios were chosen to represent the amount of instruction receivable by 
the student. The first of these is the teacher /student ratio, which gives 
an indication of the amount of personal attention the student could receive 
in a given classroom situation. The efficiency of teaching in the classroom 
is heavily dependent upon the number of students with whom a single 
teacher has to deal. Another measure of the intensity of the student's 
interaction with the forces of learning is the text/student ratio. This 
index, along with the per-student expenditure on material aids for the 
classroom, represents the effort made by the school to provoke the 
curiosity of the student and to supply an outlet for such curiosity. The 
third ratio, the number of desks per student, is intended to reflect the 
physical conditions of the classroom and the freedom of movement allowed 
the students. Current thinking in education suggests that the ideal class- 
room is one in which the teacher has a comfortable number of students to 
teach, enough texts and materials to invoke and satisfy their curiosity, 
and uncramped conditions to give students a sense of individuality and 
sufficient privacy. 

Classroom exposure time is the last of the instructional considerations 
in the submodel. If students do not have enough time to spend in classes 
which provoke their interest, then all the materials and teachers in the 
world will not produce enduring learning. Intensity of instruction must be 
coupled with extension of instruction to yield maximal results. A measure 
of the duration of instruction was thus incorporated into the submodel to 
round out the quantity of instruction index. 

As presented above, the classroom and school variables are indicators 
of the actual changes in the students' environment that are caused by the 
introduction of a Title I project. The outputs of the submodel- -changes in 
student attitudes and achievements --are based on these changes in the 
school environment and not on the absolute levels of instruction and service. 

Instruction and service indicators act as constraints on one another 
in generating estimates of improved attitudes and increased achievement. 

For some students, the principal need is for improved instruction; for 
others, increased services. More often than not, service and instruction 
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are interdependent for a given group of disadvantaged students and one 
without the other will prove futile. The Instructional Process Submodel 
accepts the school environmental changes introduced by Title I and interacts 
the characteristics of the target population with these improvements in 
services and instruction. The product of this interaction is the forecast 
of changes in attitudes and achievement resulting from the combined 
(where applicable) or separate efforts of the service and instructional 
components of a Title I project. 

The impedance portion of the Instructional Process Submodel 
deals primarily with service programs. The Submodel receives a 
description of the school environment into which proposed Title I projects 
will be introduced. Environment as used here consists of an identification 
of the particular disadvantages of the target population, and an analysis of 
the level of instruction and the services provided by the school. The 
proposed Title I project is described in terms of the changes it purports 
to make in the level of instruction and in the amount or nature of service 
provisions. Service changes directly affect student disposition toward 
learning by negating some of the ill effects of a disadvantaged background. 
This results in an improvement in the student's attitude toward school, 
and with the instructional changes that have been introduced by Title I, 
the student level of under achievement is decreased, or in other words, 
his classroom performance is improved. 

Some of the changes in the school environment may have little or 

nothing to do with the problems of the target population. Introducing 

* 

guidance counselling for students whose principal need is proper medical 
care is one possible example of the potential mismatch between project and 
problem. An important step in the Instructional Process Submodel is 
matching the individual components of a proposed Title 'I project with the 
individual components of students' negative disposition to learning. 

Projects which are inappropriate for their target populations will produce 
little change in students' negative disposition to learning and, consequently, 
will have little effect upon student achievement. 



! 



Obviously, there are upper limits on the amount of actual attitude 
and achievement change that can take place, Reducing the student/ 
teacher ratio below some low number may indeed have very little additional 
effect. Having three psychologists for every pupil would also be of 
questionable value. As a result, the Instructional Process Submode - sets 
thresholds on the various environmental variables to prevent unattainable 
achievement changes from being forecasted. The model increases 
achievement only to grade norm, so that students who achieve grade level 
averages in any of the achievement categories are automatically unaffected 
by Title I changes. This saturation at grade norm emphasizes the fact 
that the model is pointed only toward students who are underachieving 
and is indifferent to better-than-norm results. Projects which bring 
their entire target populations to grade level are considered to be of 
maximum effectiveness in the model. 

The Title* I project is described by the indicators explained in the 
previous section for the level of instruction and the services provided 
by the target school, before the project itself is introduced into the model. 

Each component of the proposed project is then added to the classroom. 

and school indicators to determine what changes will be made in the 

classroom and school environments . The individual environmental changes are 

weighted in accordance with their contributions to the variance in attitude 

and achievement change. Weighted service changes are combined linearly 

to produce the overall weighted change in service level and weighted 

instruction indicators are combined linearly to produce the overall weighted 

0 

changes *in instruction level. 

At the same time, the Title I project identifies the particular disad- 
vantages of its target population. These impedance indicators are weighted 
in accordance with their individual contribution to the student's negative 
disposition to learning. The weighted indices are then combined linearly 
to produce a "baseline" impedance for the students to be affected by the 
project. 

Using "baseline" impedance and overall weighted service change, the 
Instructional Process Submodel computes an Index of Potential Service 
Effectiveness. This index represents the amount of impedance change that 
could take place in one year if every service improvement were relevant to 
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the deficiencies of the target students. The principal assumption under- 
lying the computation of this index is that the thrust of service components 
of a Title I project is to reduce ircqpedance to learning. The computation 
further assumes that the effectiveness of service improvements is directly 
proportional to a student’s ’’baseline” impedance and inversely proportional 
to his grade level. This means that a given service component is 
increasingly effective the higher the amount of learning difficulty in the 
target population, and decreasingly effective as the grade level of the 
target population increases. Potential effectiveness is also directly 
proportional to the amount of service change introduced by the project 
component. 

Again using the individual disadvantages combined in "baseline” 
impedance and the overall weighted changes in school services, the model 
computes an Index of Service Relevance. This index represents the degree 
to which the individual service improvements match the individual disad- 
vantages of the target population. Kele vance itself is a binary concept; 
that is, a service component is either appropriate or inappropriate to an 
impedance characteristic. There are no middle values in the model. 

Expected impedance change for the target population is then computed 
in a two-step process. First, the Index of Service Effectiveness is cross- 
multiplied by the weighted Index of Service Relevance. The weight for the 
Relevance Index is the sum of the indivudual weights of each characteristic 
of impedance that is matched by a service improvement. The product of 

the two indices, Relevance and Effectiveness, is the maximum impedance 

0 

change that service improvements could achieve. Impedance represents 
student attitudes, and there is a practical limit on the amount of change 
that can occur in a year's time. The second step in the computation of 
impedance change, then, is a comparison between the Calculated maximum 

and the practicable threshold. The model assumes that this threshold on 

» 

impedance change is directly proportional to the "baseline” impedance 
level. This assumption simply reiterates the previous one that Potential 

m 

Service Effectiveness increases as the amount of learning difficulty 

I 

increases. It also implies, however, that no service improvement, at 
least in terms of the model, is relevant to students with ’’baseline” 
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impedances equal to zero (which underlines the fact that the model operates 
only on those students who have positive impedances). The threshold itself 
is called the "Practical Limit on Impedance Decline" and is described as a 
function of "baseline" impedance. The eventual impedance change forecasted 
by the model is limited to the threshold value if the product of the two indices 
exceeds it, or the -product of the two indices if they are below threshold value. 

The contribution of a change in imped ance to ultimate change in 
the achievement level of the target population is called the Service 
Fraction. The computation of this fraction assumes that achievement 
change is directly prox>ortional to the reduction in impedance and 
inversely proportional to the "baseline" impedance. The Service Fraction, 
then, varies directly as the proportional change in impedance. 

Up to this point, the description of the Instructional Process Sub- 
model has concerned itself only with the impact of service improvements 
on student impedance and the impact of impedance change on achievement. 
There are, however, instructional components to Title I projects which 
affect achievement levels directly. The conversion of these instructional 
components into contributions to achievement change proceed in parallel 
with the service computations. 

The Instruction Fraction is the contribution to achievement change 
made by Title I improvements in the level of instruction. The computation 
of this fraction assumes that changes in achievement are directly propor- 
tional to the amount of change in the instructional level. Since, by 
definition, impedance retards achievement, the Instruction Fraction varies 
inversely as the ‘'baseline" impedance of the target population. This 
means that the Instruction Fraction is the amount by which achievement 
would change if impedance were held constant. Title I projects deal 
exclusively with instructional improvements, the Instruction Fraction 
is the only agent of achievement change. Title I projects that comprise 
both instruction and service improvements combine the two fractions in 
the computation of ultimate achievement change. 

Final achievement change is computed by adding the fractional 
achievement contributions of service and instruction improvements and 
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multiplying their sum by the M achievement lag" . The "achievement lag" 
is the number of grade levels the target group lagged behind their grade 
norm in achievement. For the purposes of the Instructional Process 
Submodel, this lag represents the maximum achievement improvement 
that a Title I project can produce with a given target population. In 
most cases, the sum of the two fractions will be less than one, and some 
of the lag will remain. If the sum of the two fractions equals or exceeds 
one, then the lag will have been entirely eliminated and the model will 
saturate achievement at grade norm. As mentioned before, the Instructional 
Process Submodel does not differentiate among competing projects which 
yield achievement changes greater than the students’ lag behind grade norm. 

Below is a flow chart of the model process: 




# 






Final impedance and achievement changes are the outputs of the 
Instructional Process Submodel. Both are input into the School Flow 
Model to determine the long-range impact Title I projects will have on 
their target populations. If the same project is applied in successive years, 
the Instructional Process and School Flow models are simply iterated by 
entering the previous cycle's results for each simulated year. Changes in 
impedance and achievement are thus accumulated over time. 

Mathematical Specification of the Instructional Process Submodel 

• t 

The quantitative description of the Instructional Process Submodel 

is divided into four parts. The first part explains how the submodel 
mathematically converts the specifications of Title I proposals into their 
instructional and service components. The second part describes the 
operating characteristics of the target population. Part three describes 
how improvements in the service environment of the school affect the 
impedance of the target population. Finally, part four gives the mathe- 
matical format for computing ultimate achievement change from the change 
in impedance and the Title I changes in the instructional environment. 

Some prefatory information is helpful in understanding how the 
mathematical relationships were derived for each section of the model. 

The Office of Education classifies its Title I projects as service or 
instructional. Service projects include special classes for the handicapped, 
guidance counselling, free lunch programs and, in general, changes in 
the school designed to increase a student's health or his personal welfare. 
Instructional projects, on the other hand, concern themselves with the 
educational and cultural enrichment of their target populations. Typical 
instructional projects are special reading programs, reducing class size, 
and preschool education. In the Instructional Process Submodel, proposed 
projects are broken down into their component parts and these are assigned 
to either the classroom or school indicators. Projects that cross over the 
Office of Education classification boundary, such as tutoring for slow 
learners, therefore contribute to both the service and instructional frac- 
tions in the model. 
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Part two, which describes the disadvantaged student, consists of 
six sociological and physiological indicators which correlate highly with 
scholastic failure. The model presumes not only that the negative effects 
of these disadvantages can be neutralized but that their neutralization will 
account for most of the variance in ultimate achievement change. 

When the Title I projects interact with their target populations, 
qualitative change in the latter is assumed to take place. The size of 
the change is dependent upon the receptivity of the affected students to 
the particular environmental changes. This means that students who are 
ill-prepared to learn will be generally unresponsive to instructional changes 
such as curriculum innovations. Certain minimum changes in services, 
depending on the nature of the students' deficiencies, have to be provided 
by the school before any progress in learning can occur. Whether achieve- 
ment improves or not therefore depends upon whether the minimum 
service changes have occurred and whether the instructional changes are 
high enough to match the increase in student receptivity. 

From the above discussion, the Submodel derives the relationship 
between achievement, instructional level, and impedance which forms its 
foundation. If instruction is thought of as the driving force of education, 
impedance as the student's resistance to this force, then achievement 
can be mathematically described as the quotient of the two. Achievement 
increases as the level of instruction increases and decreases as impedance 
increases. Title I can thus affect achievement by increasing instructional 
levels, by decreasing impedances, or by both. 
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1. 0 Title I 



Each Title I project is evaluated in terms of its effect on the 
school service environment, and its effect on the level of instruction. 

Separate indices will be defined for these two effects and 
evaluated for each Title I project. 

Each index is described by two components: quality and 
quantity. Quantity is further decomposed into intensity and duration. 

1. 1 The Service Subroutine 

The Service Subroutine assigns to each Title I projett an 
index value which reflects the changes in the school service environ- 
ment caused by the project. Following are detailed descriptions of 
the two components of the service index, quality and quantity, showing 
also how they are weighted and combined. 

Quality of Service 

The first component in the service index is concerned with 
changes in the quality of service provided. Quality is measured by 
two factors: 

1. Whether or not Title I has introduced a new 
activity; and 

2. Whether or not the program is provided to the 

student without charge. 

A Title I program that upgrades or augments existing school 
services, increasing the number of guidance counselors /student for 
example, will be measured as a change only in quantity rather than as 
a change in both quality and quantity. A program providing a new 
service will, in general, have much greater effect on the target group. 
This so-called Hawthorne effect has been modelled, so that a new pro- 
gram will be more effective than an upgraded program, all other things 
being equal. 
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A positive value of one is assigned when either of the two factors 
is present; otherwise, a value of zero. The "ones" are weighted, and 
added, to yield an index of the change in the quality of service provided 
by Title I. The weighting coefficients are established on the basis of 
research findings or judgment. 

The final form of the equation is: 




P = 1: Absence of' 
prior programs 

\P = 0: Otherwise 




( F = 1: Program free\ 
to students 

F = 0: Otherwise / 



qj and 

normalize An to a value between 0 and 1. 



q’ are the weighting coefficients which also 



Quantity of Service 

The second component in the service index is concerned with 
changes in the quantity of service provided. Quantity is determined by 
changes in the intensity of service, e. g. , increasing the numbers of 
professionals /student; and by changes in the duration of service, e. g. , 
increasing the number of hours /day in which the service is provided. 

Intensity of Service 

The change in the intensity of instruction (Aj )> is expressed 

s 

in terms of three weighted ratios which define the school service en- 
vironment after Title I. Changes in the numbers of personnel, changes 
in the square feet of space available and changes in the dollars spent on 
materials are measured. Each term is then divided by the number of 
students. Each of these quotients can be related to specific Title I 
Programs such as increasing the counseling staff, expanding the size 
of plant and equipment, or providing new or improved services such 
as lunch, milk, remediation equipment and trips to museums. In 
addition, two interaction terms are provided for cases in which the 
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quality of professional service is low or insufficient in the school fi' ) 

' 4 ^ ^ 

and for cases in which conditions are cramped and materials scanty (i^), 
The final form of the equation is: 




A#'s of professionals + ^ A sq. ft. of space 

^ student 



student 



+i' 

+1 3 



A $ 



spent on materials 



student 



+ v 4 + 



The weighting coefficients (i>) normalize Hi to a value 

s 

between 0 and 1. Relative weightings for each coefficient 
are determined by research or judgment. 



Duration of Service 

The duration component measures the difference between the 
student exposure to service before and after Title I in hours/year. 
Exposure is measured in hours /day, days/week, and weeks/year. 
Each time element in then compared against a time threshold equal 
to the attention span of the target group. Projects which exceed these 
thresholds will yield diminishing returns to scale. 

The equation for the difference in duration of service is: 




s 



d' 




D* 

s 




H D W 
s s s 



where IT , and represent the thresholded values 
after Title I in hours /day, days /week, weeks /year, 
respectively, and H , D , and W represent their counter- 

o o S 

parts before Title I. 



d 1 normalizes the expression to a value between 0 and 1. 
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Index of the Value of Title I Service Offered 



Quality, intensity, and duration of service are weighted and 
combined into a value of service index (/^S ) as follows: 

As = s,Aq + s 9 Al + s. Ad 

JL S w S S 

The weighting coefficients (s). normalize A S to a scale 
of 0 to 1, in addition to providing a relative importance 
weight for each term (in accordance with each term's 
contribution to variance in impedance). 



58 

« 




-• i.v.wrrrsit- 






•a W-*’ 



TA BLE 1. 1 

The Service Submodel 






I 






Name Variable Type Range Source 













Goodness of 
service index 


A s 


Fraction 


0-1 

; 


Computed 


Service weights 


s . 

l 


Empirical 

Constants 


Depends on 
Variance 


J Civil Rights 
Survey 


Change in ser- 
vice quality 


Z_sQ s 


Fraction 


0-1 1 Computed 

I 


[Quality weights 

! 

1 




Empirical 

Constants 

i 


Depends on j Civil Rights 

Variance \ Survey 


; 

‘Absence of prior 
program 
.'Program free 
'to students 

i 

t - - - - ■ 


1 P | Measured 

I j Constants j 

! F j After I 

i ! Title I 

i 


| Oor 1 j 

j' 


| Civil 
Rights 
Survey 


[Change in 
[service inien- 
jsity 

L _ __ __ 


Ax 

s 


Fraction 


0-1 


Computed 


Intensity weights ! 

j 


| ij | Empirical 

I | Constants 


f 1 

Depends on j 

Variance j 


Civil Rights 
| Survey 


Change in #'s of | 
professionals 
Change in sq. ft. 
of space 

Change in $ spent 
on materials 

4 


1 j 

; Components ; 

of Intensity 
1 * 


i 

: Measured 

j Constants 

After 
Title I 


Fractions 


Title I data 


Difference in Ser- 
vice Duration 


Ad | 

s 


Fraction 


,0-1 


Computed 

0 ] 


Duration Weight 


d» 


Empirical j 

Constant . 1 


| Depends on j 

• Variance j 


Civil Rights j 

Survey j 


Thresholded ser- 
vice environment ] 
hours/day, days/! 
week, weeks /yr. i 
after Title I 


H' , D» W' 

\ s s' s 

1 

i 

1 


Measured 
Constants 
After 
Title I 


Fractions 


Title I data J 


Environment 
before Title I 


H s , D s , W : Measured con- 

s i stants before ! 

j Title I 

i i 

i 


< 

Fractions " ; 

• ! 
i i 


Title I data 




l 
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Summary of Equations 



1 

I 



As == s, A q + s 9 Ai + s A 

1 s 2 s 3 

Aq. = 



D 



r, 



s ~ CJ i ^ if no prior ) + q' (1 if Free) jl + (1 if parents' incosuxe —$2000)' 
program ? 5 



A I = j iAi!iL° £ P r ° ^ e 5 s ion als . , Asq. ft. of s pace A 

s J student ‘ 2 student ' 



$ spent on mat. + 
student 



Ad 



d' (H« . D* . W' - H . D . W ) 
s s s s a p. 



i 
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1. 2 The Instructional Subroutine 



The Instructional Subroutine assigns to each Title I project an 
index value which reflects the changes in the level of instruction caused 
by the project, in much the same way that the Service Subroutine assigns 
an index reflecting changes in the school service environment . 

Expected achievement improvements are dependent upon the 
quality and quantity of instruction. Title I may affect each of these 
separately or in combination. Each component is described in de- 
tail showing how it can be weighted and combined with the others into 
an index of the change in instruction afforded by the Title I programs. 

Quality of Instructio n 

The change in the quality of instruction, Aq, is decomposed into 

Index of the cha.nge in recency of 
curriculum materials measured by 
the average difference in publication 
dates; and 

Index of the change in teacher quality 
measured by the change in teacher 
experience in years. 



two factors: 

ArEC: 



Ao t : 



Each of the two factors is weighted according to the achievement 
variance accounted for. "Recency'’ and teacher "quality" are not linearly 
related. For example, when curriculum materials are antiquated, the 
effectiveness of the teacher becomes even more crucial in the students' 
education. Thus, the change in the quality of instruction Aq» is dependent 



not only on changes in recency and teacher quality, but also on the 
interaction of the tv/o, when recency is less than a threshold 
The final form of the equation is: 



Aq = q x (ArEC) + q 2 . Aq t 



1 + 



1 if ArEC 2«( 
10 Otherwise 



The weighting coefficients (q) normalize Aq to a value between 
0 and 1, in addition to accounting for the variance associated with 
each term. 



Quantity of Instruction 

The second component in the instructional index is the quantity 
of instruction provided. Quantity is determined by changes in the 
intensity of instruction, e. g. , increasing the number of teachers/ 
student, and by changes in the duration of instruction, e. g. , adding a 
remedial reading program. 

Intensity of Instruction 

The change in the intensity of instruction, (/\l), is expressed in 
terms of four weighted ratios which define the instructional environment 
after Title I. Interaction terms are used when teacher mastery is low 
(i 5 ), when classroom conditions are cramped and instructional materials 
scanty (i^), and when the class size is too large or the number of teachers 
is inadequate (i^). 

The final form of the equation is: 

A- , A £ 's of teachers . A#' S of texts . A#' S of desks 
1 student x 2 strident + 1 3 student 

+ i 4 A$ spent on aids , . . . 

student *5 + 1 6 * 1 7 

The weighting coefficients (i) normalize Al to a value between 

0 and 1. Relative weightings for each coefficient are de- 
termined by research or judgment. 

* 

Duration of Instruction 

The duration component measures the difference between student 
exposure to instruction before and after Title I in hours /year. Exposure 
is decomposed into hours /day, days /week and weeks /year. Each time 
element is compared against a time threshold equal to the attention span 
of the target students. Projects which exceed these thresholds will 
yield diminishing returns to scale. 

The equation for the difference in duration of instruction is: 

Ad = d (H‘ . D' . W' - H. D . W) 
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where H', D 1 and W' represent the thresholded values after 
Title I in hours /days, days/weeks, and weeks/year respectively, 
and H, D, and W represent their counterparts before Title I. 
d normalizes the expression to a value between 0 and 1. 

Index of the Change in Instruction 

Quality, intensity and duration arc weighted and combined into 
an index of change in instruction { /\C) for each category of achievement 
(j) as follows: 

Ac. = c (Aq) + c 2 (Ai) + c 3 (Adj 

J j j j 

The weighting coefficients (c.) normalize Ac to a scale of 

3 

0 to 1, in addition to accounting for the variance in each 
achievement category (j). 



STABLE 1. 2 



The Instructional Subroutine 



Name 



Variable 



Type 



Range 



Source 



Change in instruc- 
tion by Title I 


Ac. 

j 


Fraction 


— 
0 - 1 


Computed 


Achievement cate- 
gory weights 


C. . 
h J 


Empirical 

constant 


Depends on the 
amount of vari- 
ance accounted 
for each factor 


Civil Rights 
Survey researc 
findings 


Change in instruc- 
tional quality 


A Q 


Fraction 


0-1 


Computed 


Quality weights 




Empirical 


Depends on 
variance 


Civil Rights 
Survey re- 
search findings 


Recency threshold 




Empirical 


Depends on 
recency 


Civil Rights 
Survey 


Index of the change 
of curriculum 
materials I 
Index of the change 
in teacher quality 


Arec, 

Aq t 


Measured con- 
stants after 
Title I 


Fractions 


Title I data 


Change in instruc- 
tional intensity 


Ai 


Fraction 


0 - 1 


Computed 


Intensity weights 


i. 

l 


Empirical 

constants 


Depends on 
variance 


Civil Rights 
Survey 


Number teachers/ 
student 

Number texts/ 
student 

Number desks/ 
student 

Dollars spent on 
aids / student 


Components 
of Intensity 


Measured con- 
stants after 
Title I 


Fractions 
* ' 


Title I data 

i 


Difference in instmc t 
tional duration 


Ad 


Fraction 


0-1 


i 

Computed j 


Duration weight 


d 1 

i 

» 


Empirical 

constant 


Depends on 
variance 


Civil Rights ] 

Survey ! 


Threshold in- 
struction en- 
vironment hours/ 
day, days /week, 
weeks /year after 
Title I 


H», D», W» 


Measured con- 
stants after 
Title I 


Fractions 


Title I data 

■ i 


S^me environment 
before Title I 


H, D, W 


Measured con- 
■' stants before 
Title 1 


Fractions 


Title I data 

•V 






c 



Summary of Equations 



Ac ; 


= Cj Aq + c 2 Ax + < 


"3. 


j j 


J 


a 

<3 


= q t AREC+q 2 AQ t 


r 

1 + 

M 


/\ x = 


/Sir's of Teachers 






i student 


x 2 



Ad 



f l if A:REC2=tf 


*» 




0 Otherwise J 


- 




A#'s of texts 


+ 1 J 


A H's of desks 


student 


+ 13 


student 



• -Yi 



spent on aids 



student 



+ i 5 + i 6 + i? 



ti\ D = d(H' . D' . W* - H . D . W) 
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2. 0 The Disadvantaged Student 



The target population is characterized by a set of six 
factors of disadvantage which, when aggregated, constitute its "base- 
line" impedance to instruction. This impedance is expressed as an 
index of values between 1 and 10 and is denoted by the letter Z. When 
operated upon by appropriate service components of a Title I project, 

Z decreases and this decrease becomes the service contribution to 
achievement change. 

Baseline Calculations: The Index of Students' Impedance 
Instruction 

A particular student population is targeted for a Title I 
project. This population is characterized by certain educational de- 
ficiencies which are referred to as the students* "baseline impedance. " 
Impedance may be academic, psychological, sociological, or combina- 
tions of these. In all cases, however, it represents the students' re- 
sistance to instruction. The Title I project is intended to alter the stu- 
dents' scholastic environment in such a way as to offset these existing 
disadvantages. Changes are made in the quality and/or quantity of in- 
struction, in school services, or in both at the same time. What is 
expected is a reduction in the students' achievement gap and a reduction in 
their impedance to instruction. Impedance will be measured for each 
of the student types (t) comprising the target population. These baseline 

(B) values will be symbolized as: Z_ . 

±5, t 

Impedance to instruction is decomposed into the following 

six factors: 



$ 


Income of Parent 


($2000 or less, or other) 


E 


Parents' education 


(Elementary or other) 


P 


level 


H 


Handicap 


(Physical, mental or 
emotional) 


S 


Solidarity of family 


(Disrupted or intact) 



See Benjamin S. Bloom, Stability and Change in Human 
Characteristics , New York: Wiley, 1964; especially Chapter 4. 
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Grade /Achievement lag for the cohort. The 
cohort contains all the students in the same 
class, or in the same grade, in the target 
school. 

Grade /Achievement lag for the student in the 
basic skills of reading, language and math. 



Each of these factors is converted to an index value of 0 or 1 
and then weighted by the amount of variance it accounts for in determining 
changes in achievement. The index of impedance to instruction ranges 
from 1 to 10 and is described by the equation: 



Z 

B, t 



= 1 + (a) ( 2 j) 



+ ( z ?) 



1 if Parents’ income isZz $2000 
0 Otherwise 

1 if Parents’ education is elementary] 
0 Otherwise / 






+ (z 3 ) 



1 if Handicapped 
0 Otherwise , 



+ < z 4 ) 



1 if family disrupted 
0 Otherwise - 



+ ( z 5 ) 



1 if cohort lag is ~ 3 
0 Otherwise J 



+ (z 




1 1 if student lag is ~~ 3 
10 Otherwise 1 

'i 



1 + 



1 if Parents' 
0 Otherwise 



education is 



elementary 







/ 

normalizing the term. 



If tne students' parents have less than a secondary education, the 
significance of the students' achievement lag in the basic skill areas is 
increased. With poorly educated parents, a student's need for achieve- 
ment is far less likely to be high. * 

These interactions between factors are accounted for by including 
(as part of the sixth term in the impedance equation) an additional element; 

*R. H. Dave, The Identification and Measurement of Environmenta l 
J-jiPcess Variables that are Related to E ducational Achievement, Unpublished 
Ph.D. Dissertation, University of Chicago, quoted in Bloom, Ibid. , p. 124. 
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(z 6 ) (1 if student lag is .*£ 3) (1 if parents' education is elementary) 



The value 3 is derived by using research results that show a lag 
of one grade level by the third grade, two by the sixth grade, three by 
the ninth, and four by the twelfth are on the critical path toward failure. 
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TABLE 2. 0 



Index of the Student's Characteristic Impedance to Instruction 



Name 


Variable 


Type 


Range 


Source 


Index of Impedance to 


Z B t 


Index 


1 - 10 


Computed 


instruction 




value 






Normalizing constant 


a 




Depends on the 
sum of z weights 


Computed 


Impedance component 


Zj to z^ 


Empirical 


Depends on the 


Civil Rights 


weights 


constant 


amount of vari- 
ance accounted 


Survey 










for each factor 




Income of Parent 


$ 


Measured 






Education level of parent 


E 

P 


constants 




School 


Handicap 


H 


before 


0 or 1 




Solidarity of family 


S 




Grade /lag for cohort 


L 

c 


Title I 




Data 


Grade /lag for student 


L 

s 









Summary of Equations 

Z = 1 + a ^ z. D. + (1 if L g ^ 3) [(1 if E p = elementary) + l] 

i=l 

where D. are the disadvantage- factors. 
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3. 0 The Model Environment 

In this section the amount of disadvantage removed by the Title I 
project is computed. The equations involve the conversion of the Title I 
service index, (As), into an index of impedance change, (Az). Three 
concepts are involved: 

1. a measurement of the effectiveness of service 
(quantity); 

2. a measurement of the relevance of that 
service to reducing the disadvantage 
(quality); and 

3. the combination of quantity and quality 
into an index of impedance change 
limited by a behavior threshold for the 
target group. 

The following assumptions are made: 

1. The effectiveness of a Title I service program 

is reduced when the program is introduced in .the 
later grades;* however, this effect is lessened 
when the student impedance is high. 

2. Different Title I service programs tend to re- 
duce the factors of instructional impedance 
differentially; i. e., the programs operate to 
different degrees upon students having different 
impedance makeups.* 

3. The amount by which the student impedance 
can change in any one year period is dependent 
upon the student impedance level. As a corol- 
lary to this assumption, there is no Title I pro- 
gram which is relevant to a student possessing 
no characteristics of disadvantage. 

3. 1 Service Effectiveness 

The service effectiveness factor is computed by combining the 
potential service provided by Title I with the particular target group it 
affects. Assumption 1 (see section 3. 0) asserted that the effectiveness 
of Title I service will be reduced at higher grade levels. This reduced 
effectiveness should not be confused with the factors making the Title I 
program relevant to the student's particular disadvantage. (See sec. 3.2) 



♦ Bloom, Ibid. , pp. 125-126. 



In graphical terms, the potential effectiveness of a given Title I service 
component drops off with grade at an accelerating nace: 




When the target student has a high level of impedance to instruction, 
he is assumed to be more receptive to a given Title I service improve- 
ment regardless of when it is applied. 

These two effects apply differentially to students with different 
degrees of disadvantage. Effectiveness is reduced most for students 
with few disadvantages at the latest grades: 




, high degrees of 
disadvantage 

few disadvantages 



The equation for the service effectiveness factor represents 
the quantity of additional service afforded a specified target group by 
Title I. The numerator, /\s, the value of service index, is a function 
only of Title I and was described in section 1. 1. The denominator is a 
function of the impedance of the target group (/\z), described in section 
2, and the target group grade. 

The equation for the service effectiveness factor is: 




% 

i 
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3. 2 Service Relevance 

Assumption 2, of section 3. 0, stated that different Title I pro- 
grams would produce different effects on different students. In the 
model, the quality of the Title I school service, as it relates to the tar- 
get population, is determined by a set of relevance numbers (0 and 1) 
which indicate the specific factors of disadvantage directly affected by 
the Title I program. Each Title I service program, whether free lunch, 
professional service, or special programs for the handicapped, will 
attempt to attenuate the characteristics of disadvantage to different 
degrees. 

Two factors are involved: the specific characteristics making up 
each student’s impedance and secondly, the specific impedance char- 
acteristics to which the Title I program is relevant. When the two 
sets of factors are identical, the relevance of service will be at a 
maximum; when they are disparate, the relevance approaches zero. 

3. 3 The Change in Impedance (Az), caused by Title I. 

When the two factors, service quantity and quality, are combined, 
they produce an index of impedance change, Az, caused by Title I. 
Relevant (R) components of impedance are weighted and added together. 
After being normalized, this sum is multiplied by the service effec- 
tiveness factor (E g ) to produce an index of the change in impedance (A Z). 



Az = (a) (Ej< 





r 


,1 if Parents' income isf'$2000 i 


R 1 


z i 


^0 Otherwise 


k 







R. 



z 



'2 



R 



z. 



1 if Parents’ education is elementary 
0 Otherwise 

) 

1 if family disrupted 1 



1 if handicapped 
^0 Otherwise 



+ R, 



\0 Otherwise 



R 



R, 



1 1 if cohort lag is —3 
5 \0 Otherwise 



1 1 if student lag is ~ 3\ 
^ \0 Otherwise / 






l + 



fl if Parents' ed. is elem.j 
i0 Otherwise 






where: E is the service effectiveness factor (section 3.1) 
s 

R to K, are the relevance numbers (section 3. 2) 

1 6 

The multiplier (a), normalizes the term to a value between 0 and 9 
(see section 2. 0 for detail). 

z to are the same disadvantage factor weights 
described in section 2. 0 



The relevance numbers R; to R 
program in terms of which impedance 



^ are determined by the Title 
factors the Title I project can 



X 



overcome. 
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^ Allo wable One- Year Change in the Impedance Value 

Caused by Title I 

Assumption 3, section 3. 0 implied that there was an upper bound 
on the amount of impedance change that could be expected in the target 
population after a one- year exposure to a Title X program. It is 
further assumed that this upper limit is directly related to the baseline 
impedance. The greater the impedance, the greater the potential 
impedance change, all other things being equal. 

Included m assumption 3 is a corollary stating that there is no 
Title I program which is relevant to a student having zero impedance. 
This assumption is necessary in the model because a bounded index 
scale is needed to prevent Title I from causing impedance to go out- 
side the index scale. The boundaries are Z . and Z 

min max. 

A student possessing all six disadvantage factors is subject to 
the maximum rate of impedance decline. Lesser "baseline" impedance 
yield lesser maxima. 



Max = 10 

degree of 
disadvantage 

Min = 1 






years 



The above curve approximates the removal of one factor of 
disadvantage per year. 

0 

The equation describing the maximum rate of decline is: 

Z - Z . 

M= - 



1 *2 Z 2 
max 



Z . = 1 
min 



Z =10 
max 



* 
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3. 5 Final Value for the Change in Impedance 



As a final step in computing the reduction in impedance by the 
Title I project, the index value, Az, computed in section 3. 3 must be 
limited by the boundary value M computed in 3. 4. 

A Z is compared against the Value M. WhenAz is larger than 
M, it is changed to equal M. The need for this change will probably 
be greatest when the students 1 impedance is low. 
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TABLE 3. 0 

The Model Environment 




Service Effective- 
ness Factor 


E 

s 


Fraction 


0-1 


Computed (3.1) 


Index of the good- 
ness of Title I 
service 


S 


Fraction 


0-1 

i 


Computed (1. 1) 


Index of Im- 
pedance 


Z 


Index value 


1-10 


Computed (2.0) 


Change in Im- 
pedance caused 
by Title I 


z 


Index value 


1-10 


Computed 
(3.2 or 3.4) 


Normalizing 

constant 


a 




Depends on 
the sum of 
z weights 


Computed (2.0) 


Relevance 


R, to R^ 


Measured 

constants 


0-1 


Empirical Re- 
search or 
judgment 


Impedance 

Component 

Weights 


Zj to z^ 


Empirical 

constants 


Depends on 
the amount of 
variance 
accounted for 
each factor 


Civil Rights 
Survey 


Income of Parent 
Education level of 
parent 
Handicap 

Solidarity of Family 
Grade /lag for cohort 
Grade /lag for stu. 


$ 

E 

P 

H 

S 

L 

L c ' 
s 


Measured 
constants 
before 
Title I 


0 or 1 


School Data 


Maximum allowable 
one- year change 


M- 


Fraction 


0-1 


Computed (3.4) 



E s = 



A 



Summary of Equations 



1 + 



M 



Z = a E 



s 



' 5 * 

-Y- ! 

L R.z.D. + R, z, (1 if L ^ 3) 
i=l ill 66 s 



D i are the characteristics of Impedance 



(1 if E = elementary) + 1 



M = (Z-l) 



6 fl +- 



Z2 
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4. 1 The Change in Achievement 

Title I is described in terms of instructional and service changes. 
Either, or both, of these kinds of changes may be present in any Title I 
project. It is now assumed that the achievement lag can be attributed 
to the students’ impedance and to the level of instruction itself. Adverse 
effects on achievement due to impedance factors can be reduced by 
properly applied and relevant Title I service components, while the 
adverse effects of poor instruction can be reduced by properly applied 
and relevant Title I instructional components. 

This leads to the following relationship: achievement is directly 
proportional to instruction, and inversely proportional to impedance. 

Two fractions are defined to represent the contributions of service (F ) 
and instructional (F^) improvements. 

Instruction Fraction 

Achievement = C/Z, where C describes the level of instruction 
and Z the student impedance to instruction. F^ is the fractional in- 
struction change caused by Title I. 




A C is the index of instruction from section 1. 2 
Service Fractions 

To find the corresponding change in achievement caused by reduced 
impedance, the difference before and after Title I is computed: 






F 

s 



C 

(Z - A Z) 



£ 

Z 



This equation can be rearranged to read: 

r c Az 

s Z Z - A Z 
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4. 2 The Change in Achievement 

Final achievement change is: 

Aa = (F. + FJ . Lag 

where "Lag" is the number of grade levels the target group lags behind 
grade norm in achievement, and represents the model's upper bound 
on possible performance improvement. 

4. 3 The Model and Time 

This description of the Instructional Process Model assumes that 
calculations are made for a one year period. In situations where a Title I 
project acts on an entire school system, or for a period of successive 
years, the Instructional Process model must be re-entered each year 
with new data, describing the target group's previous achievement lag. 

The feedback of these data each year allows computation of the cumulative 
change in achievement. 
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TABLE 4. 0 



The Change in Achievement 



.Name 



Variable 



Type 



Range 



Source 



The Change in 
Achievement 


Aa 


Forecasted 

Variable 


0-8 grade 
levels 


j Computed (4. 2) 

| 


jin d ex of the Change 
an Instruction Caused 
by Title I 

t— 


Ac 


Index 

• 


0-1 


Computed (1. 2) 


\ 

Change in impedance 
paused by Title I 

4 — 


A z 


Index 


1-10 


Computed (3, 3) 

I 


1 

jlndex of student 
Impedance to in- 
struction 


Z 


Index 


1-10 


' 

Computed (2. 0) 


Student achievement j 
Lag 


Lag 


Measured 
Constant 
before 
Title I 


0-8 grade 
levels 


School Data 


Fractional instr. 
change 

Fractional ser. 

( change 


01 


Fraction 


0-1 


Computed (4.1) 









Summary of Equations 



Aa = 



A 



+ tag | 



, z -A z j 



Gap = (F. + F s ) 



Gap 
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CHAPTER VI 



THE SCHOOL FLOW SUBMODEL 



This chapter describes a mathematical model which predicts the 
achievement levels of students at each grade in their school careers 
subsequent to the grade of Title I intervention. Evaluation of the immediate 
effect of a proposed Title I program is discussed in Chapter V. The pro- 
cedure in this chapter extrapolates achievement forward in time from the 
point at which the Title I program is no longer present in the school. By 
comparing the achievement levels extrapolated from the effects of a 
proposed Title I program with those extrapolated without a Title I program, 
we can assess the differential effects of the proposed assistance. 

The model uses conditional probabilities to predict the achievement 
pattern in a given grade from the achievement pattern in the previous 
grade. The probabilities are of the form: "Given that a student in the 
fourth grade passed English, social studies, and science, and failed 
mathematics, the probability that he will pass English, social studies, 
science and mathematics in the fifth grade is 0. 40. " These probabilities 
are defined for each grade -to-grade transition, and for all combinations 
of subject passes and failures in each grade, and all combinations of 
passes and failures in the following grade. The number of students passing 
any combination of subjects in the following grade is predicted from 1) 
the transitional probabilities for the preceding grade and 2) the number of 
students passing each combination of subjects in the current grade. Thus, 
one can observe the probable future consequences of early failures --shown 
conceptually in Fig. 1. 

The School Flow Submodel accepts the immediate achievement changes 

0 

in a target student population resulting from an education improvement 
program, and propagates these "local" changes ahead in time to dropout, 
or to course-of-study selection and graduation. It thus converts short- 
term student achievement changes into long-range forecasts of changes in 
achievement and number of dropouts and the number and quality of graduates 
Individual subject-grade failure interdependencies in the curriculum 
matrix, such as the probability of a student failing third-grade science if he 
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has failed second-grade reading for example, have been derived from 
several hundred student records of multiple failures. In a significant 
percentage of the cases examined, failures (achievement gaps) "spread" 
from one or a few subjects to additional subjects downstream. We 
reproduce this indicative relationship in the School Flow Submodel to 
propagate the effects of early failures, and correspondingly to propagate 
the reductions in early failures resulting from Title I improvement 
programs in terms of reductions in later failures, dropouts, and low 
achievers . 

It is especially desirable to measure the change due to a Title I 
program in student's achievement patterns throughout their elementary 
and secondary school careers. The information gained from a grade-by- 
grade indication of potential changes in achievement levels can provide 
policy planners with better insight into the effects of Title I programs. 

It is important to know, for instance, not only that a program applied 
during the second grade has no residual effect remaining by high school 
graduation, but also that the program has only marginal effect on achieve- 
ment after the fifth grade, while another program has potentially as strong 
an effect through the eighth grade. Grade -by-grade achievement records 
are also useful in estimating and predicting dropout and truancy rates. 

As shown in Chapters VII and VIII, these predictions rely heavily on 
achievement measures. Educators and analysts, because of their familiarity 
with grade-by-grade achievement data, should be able to make good use of 
the grade -by-grade achievement projections and be comfortable with infor- 
mation in this form. 

In this chapter, we are concerned with the average student of a par- 
ticular "student type". Student types may be defined differently for each 
community for which the model is used. Classifications may include 
.income and ethnic background. All the students being simulated are placed 
in one of up to four student types. The typology is chosen to reflect the 
possibility of differential effects on the students as the result of the Title I 
program. The differential effects will be related either to the students' 
achievement and personal characteristics, or to their sociological back- 
grounds. 
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After students are classified into these four types, the population size 
of each type, and the background characteristics and achievement level of 
an average student of each type are determined. This determination may 
be made through the use of expert judgment or by statistical methods. 

In most school systems, it is likely that the former method will be used. 

For more on this subject, see Chapter V. Within each student type, we 
can construct achievement distributions for the students in that type. 

Thus, we can use average characteristics to give us an indication of the 
statistical properties of the achievement levels of students of each type. 

It is important to describe explicitly the measurements of achievement 
which are used. We consider each subject to be graded independently of 
other subjects, although there may be high correlation between levels of 
achievement for different subjects. We also assume that a pass -fail 
threshold has been selected for each subject in each grade. The threshold 
may be stated in terms of a percentile score on a standardized achievement 
test, in terms of numerical or alphabetical grade averages, or in terms 
of subjective teacher judgment. With this threshold, one can classify 
students into pass -fail groups for each subject in each year. A measure of 
achievement for a type is based on the number of students in that type in 
each of the pass -fail combinations over all subjects. For example, if only 
two subjects are considered, mathematics and English, then achievement 
is described by the set of four numbers indicating the number of students 
who pass both subjects, those who pass mathematics and fail English, those 
who pass English and fail mathematics, and those who pass neither. It is 
also possible to compute the expected number and standard deviation of 
subjects passed. 

The projections of achievement from the current year to the following 
year depend on the achievement during the current year. We compute the 
probability of passing some subjects and failing others in the next grade, 
given a particular pass -fail combination in the current grade. For example, 
consider the two-subject case just described. We compute the probability 
of passing both subjects in the next grade, given that both are passed in the 
current grade, the probability of passing only English in the next grade, 
given that both are passed in the current grade, etc. 
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These probabilities are multiplied by the number of students passing 
each combination of subjects in the current grade to give the number of 
students passing a given combination in the next grade. For example, 
the number of people passing mathematics and English in grade 4 is equal 
to the number passing both in grade 3, multiplied by the probability of 
passing both subjects in grade 4, given that both were passed in grade 3, 
plus the number of people passing only mathematics in grade 3, multiplied 
by the probability of passing both in grade 4, given that only mathematics 
was passed in grade 3, etc. 



Current Grade 
Number of 
students 
passing and 

failing 

combinations 
of subjects 



(operated on by) 



Set of 
Conditional 
Probabilities 



Next Grade 
Number of 
students 
passing and 
, — ^ failing 

combinations 
of subjects 



Figure 2 

A change in achievement in the current grade due to a Title I program 
is assumed not to affect the probability that a student, having passed and 
failed combinations of subjects in the current grade, will pass and fail a 
combination in the following grade. 

The set of subjects which a student will pass in the following grade is 
dependent only on the set of subjects passed in the current grade. The effect 
of an increase (or decrease) in the average number of subjects passed by 
his peers is not considered. * 

Based on the above measures and assumptions, a recursive scheme 
for projecting achievement has been adopted. The number of students passing 
each combination of subjects for each grade is computed from school records, 
as is the set of probabilities at each grade specifying the relationships 

i 

between subjects passed in the previous grade and subjects passed in the 
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current grade. The numbers of students passing each combination of 
subjects in the grade at which the Title I program is to be applied is 
determined from the student distribution before Title I and evaluation of 
the Title I program in terms of its effects while present in the school. 

This new set of proportions is then combined with the set of probabilities} 
specifying the relationships of subjects passed between grades, ana new 
numbers of students passing each combination of subjects are determined, 
starting from the grade at which the Title I program was applied and 
working up to graduation. At each grade, the expected number of 

subjects passed, and its standard deviation can be computed. 

Once data are gathered on the conditional probabilities of the average 
student’s passing a certain combination of subjects in a given grade (given 
that he passed a fixed combination of subjects in the previous grade), this 
information can be used to project achievement in each grade of any set 
of students, actual or simulated, whose performance in the previous 
grade is known. 

Description of a mathematical projection method like the one presented 
here is only a first step towards implementation. Parameters used in the 
procedure must be estimated from available data and the assumptions 
underlying the model must be validated before it can be used for actual 

projections as inputs to decision-making. 

Estimation of the parameters involves estimation of the conditional 
probabilities of passing a set of subjects in grade t, given that a set of pre- 
requisite subjects were passed in grade t-1. The common sense estimate 
is the ratio of the number of students who, having passed the set of pre- 
requisite subjects in grade t-1, passed the desired set of subjects in grade t, 
to the total number of students in grade t-1. This estimate is shown in 
Note 1 to have certain desirable statistical properties. Certain tests can 
be made from the estimates: a test that the conditional probabilities are 
the same for each grade, and a test that the conditional probabilities are 
equal to specified constants. The results of these tests are described in 

Note 3. , 




In many schools, data may be incomplete or non-existent. The 
common sense estimates described above are not useful in this case, 1 

because many or all of the ratios are zero. Even if provision for additional 
data gathering is made, some time elapses between the commencement of 
a data gathering effort and its completion. In order to implement the model 
before and during the data- gathering efforts, estimates are required 
which combine whatever data is available with a priori knowledge. The 
Bayesian estimation procedure described in Note 2 defines an estimate 
which combines a priori , subjective evaluation of the conditional proba- 
bilities with whatever data is available. As more data become available, 
they can be incorporated into the estimates, with a very large sample of 
data, the initial subjective evaluations play an almost negligible role in 
determining the estimates. 

One of the most important assumptions underlying the procedures 
described in this chapter is that only the achievement record of the current 
year is necessary to predict achievement in the next year. An experiment 
to validate this assumption is described in detail in Note 3. The results, 
although based on too small a sample to be conclusive, are encouraging, 
i. e. , assumption seems justified. It is expected that further validation of 
this assumption will be attempted before the procedure is fully implemented. 

In order to describe the approach discussed here in quantitative 
terms, it is helpful to present a brief introduction to the theory of Markov 
chains. Markov chains, by virtue of their simplicity and flexibility, play 
an important role in applications of probabilistic processes in many areas. 

A good introduction to Markov chain theory is found in Kemeny and Snell (3). 

The principal relevant elements of the subject are summarized here. 

Consider a quantity which takes on one of a finite set of values at a point 
in time, t; its value may change at fixed discrete points- in time, t, t+1, . . , 

These values may be descriptive, or they may be quantitative. For example,' 
assume that the brand of soap purchased by a consumer is the variable of 
interest. Assume further that the consumer purchases one bar of soap 
every week, and that there are M brands available. The variable "bar of 
soap brand name" takes on integer values from 1 to M, according to the 
brand purchased, for each week. Purchasing a bar of the i** 1 brand of 
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soap is equivalent to state i, written s.. There are, at each week t, states 
Sp s 2 , s 3 , ...» s m which are then possible. 

The central assumption of Markov chain theory states that, to deter- 
mine the probabilities of the various states at time t, given the states at 
the previous weeks t-1, t-2, etc. , only the state of the previous week (t-1) 
must be studied. According to the Markov Assumption, information about 
the states in earlier weeks does not change the conditional probability of 
moving from a state at time t-1 to a state at time t. In order to formalize 
this assumption, let P^(s^) be the probability of the random variable's being 
in state s. at time t. In the example discussed, p t (s«) is the probability 
that a consumer buys a bar of Brand i soap during the t tn week. This 

probability can be loosely interpreted as the fraction of all consumers who 

til 

buy Brand i soap during the t week. 

The conditional probability P (sjsj, t) is defined as the probability , 
that the random variable is in state Sj at time t, given that it was in state 
s. at time t-1. By the laws of conditional probability, 



P(s jSi ,t) = 



P(sj at t and s- at t-1) 



P(s. at t-1) 

J 



The conditional probability equals the ratio of the joint probability of being 
in state s. at time t and in state s^ at time t-1 to the probability of being in 
state s^ at time t-1. In terms of the example, the conditional probability 
can be interpreted as the proportion of consumers who, having bought 
Brand i during the (t-l)^ week, buy Brand j during the t^ week. This 
proportion is numerically equal to the ratio of the number of persons who 

til til 

bought Brand i during the (t-1) 1 week and who bought Brand j during the t in 
week. 

Using the laws of conditional probability, one can* show that 



M 

P t (s.)= ^ P(SjS i( t) P^te.), for j=l, . . . 



M 



or, in matrix notation, 

| P t = P t P t-1‘ 

This equation can be used recursively to show that 

p t+ i = p t=i p t = p t=i p t Pt-i ■ 




i 
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etc.. , so that 



ip = p p 

t+k t+k t+k-1 



p t p t- i 



The key assumption underlying the Markov model is that the conditional 

transition probabilities P(s.s.,t) are independent of the probabilities of 

J 

state at time t~2. This assumption may be written: 



P(s . at t 
V J 



s • at t — 1 , Si at t — 2 , • • 
l k 



= P(s . at t, 
J 



s. at t-1) 
i 



— P (s ., s . j t). 

J 



Validation of this assumption is crucial to a correct application of Markov 



chain theory. 

It is also important to note that, while the Markov property may hold 
for the M states as they are defined, it does not generally hold for a subset 
or combination of the M sets into M’ ^ M states. Hence, the assumption 
above must be verified for each definition of states studied, even if only two 
states are combined or one is deleted. 

Markov models in educational modeling have been used extensively, 
both for individual student models and for aggregate models. For a dis- 
cussion of the former, see Bush and Mosteller (2). Markov chain theory 
is used here to define the recursive projections of achievement. 

For a given school, divided into T grade classifications, and offering 
r subjects, there are assumed to be pass -fail thresholds for each subject. 

A student takes all of the r subjects in each grade classification 1 through 



T, and receives a pass or fail grade, receiving r* T grades in all. His 

academic status at the end of the t^ grade can be expressed in terms of 

a state s. where i is a binary number of r digits, and a M one M indicates pass 

in the subject corresponding to the place of the one in the binary number. 

If, for example, r=3, and the student passed the first and third subjects, 

then he would be in state s! »i . The possible states for this example are 
t _t _t j, A t t t 



s 



111 



s 110’ s 101 



oil* 100’ 010’ 001 ’ 000* 



It is clear that there are in general, 2 r states. If two thresholds 



implying three classifications, good, pass, and fail, were used, then the 
states could be defined similarly, and there would be 3 states. If we 
assume that the Markov assumption holds, i. e. , that academic performance 
in grade t depends only on that of grade t-1, and not directly bn anything 
earlier, then a matrix can be defined to predict educational performance. 
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Suppose that the student has a certain probability of passing each of 

the r subjects which is set as an initial condition, i. e. , P(s?.^. m P(s^)) 

r t o ^ 

for all of the 2 states. This probability is written as p. , p., and 

the vector of probabilities corresponding to the 2 states as P . If the 

conditional probability state vector at time t could be constructed directly 

by the relation, 

p t = P t P t-1 ’ 

where P is a matrix whose elements are the conditional probabilities 
t t- 1 

described. Specifically if s. is a state at grade t-1 whose subscript 

index represents a binary number, with ones corresponding to subjects 

passed, and s^ is a state at grade t whose binary subscript also indicates 

1 

a configuration of passed subjects, then p. - (t) is an element of P, defined as 

ij .i 

P..(t) = Prob (s^ s!” * ). 
ij J i 

By applying this relation repeatedly, we may write p^. in terms of the 
transition matrices and the initial conditions as: 




P P 
t t-1 




If the product of transition matrices 




t 

£ 



P. 

J 



is computed, then different initial conditions can be used to determine the 
final state probabilities. Alternatively, if the final state probability vector 
is known, then initial conditions can be calculated, using a matrix inverse 
(if it exists), as 

• Po = < P t )' X Pt' 

and different end conditions can be studied in terms of their effect on the 
initial probability state vector. If the Markov chain is stationary, then 

T 5 

P..(t) = p. . for all t=l . . | , T, and Pfj. = P , the T power of the one-step 
transition matrix. 



t 
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Example: Suppose, m grade 2, only mathematics and English are 
offered. Suppose, out of 50 students, 10 pass both, 20 pass only English, 
10 pass only mathematics, and 10 fail both. Suppose that the conditional 
transition matrix is, with M standing for "pass math" and E symbolizing 
"pass English, " 



(grade 2) 



ME E M NONE (grade 3) 



ME 


.6 


.2.2 


0 


E 


. 2 


.4.2 


. 2 


M 


. 2 


.2.4 


. 2 


NONE 


0 


.2.2 


.6 



and it is desired to predict the achievement at grade 3. The solution is given by: 



.6 
. 2 
. 2 
0 



.2 

.4 

.2 



. 2 
. 2 
.4 
. 2 



0 

. 2 
. 2 




X 




so that 12 pass both subjects in grade 3; 14 pass English only; 12 pass 

math only, and 12 pass neither. 



The state probabilities at each grade can be used to compute the 
average and standard deviation of the number of subjects passed, which 
give some idea of the effect of Title I programs on the base-line 
achievement. 

Each of the 2 r states has a binary number of r digits associated with 
it. Let the function Z(i) be the sum of the digits of the binary number i. 
For example, if i= 1 0 1 , Z(i) = 2. The average number of subjects passed 

per student in grade t can then be computed as: 

r , 2 r 

Z = average subjects passed^ = I P t (s.) Z(i). 

i=l 

The amount of grade advancement can be defined as the ratio of the 
average number of subjects passed to r, the number of subjects offered, 



* 
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and expected achievement at grade t can be defined by the recursive relation: 

r 



A t A t-1 + 



P t (s.)z(i)/r 



i=l 



The standard deviation of the number of subjects passed is given by 




and this number is the standard deviation of grade achievement at grade t, 
conditional on the achievement at grade t-1. It will be assumed that 
the distribution of achievement at grade t is normal, with parameters 
as shown, conditional on the achievement at grade t-1. 
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Note 1. Classical Inferences for Markov Chains 

In this note, estimation of the necessary parameters, the elements 

of the transition matrices P , is discussed. The classical statistical 

technique of maximum likelihood estimation is described, computational 

formulae are presented, andlcsts of hypotheses are given. Most of the 

work concerning maximum likelihood estimation and hypothesis testing 

is presented by Anderson and Goodman (1). A good introduction to 

these topics is presented in Mood and Graybill (4). 

Most school districts have large numbers of grade histories extending 

over the period [1, T]. Any one student is represented over the T grade 

12 T 

periods by a vector of states (s. , s. , , s^ ), where i, j, and k 

are binary numbers representing a pass -fail configuration. By sampling 
the records of the school, it is possible to estimate p..(t), the elements 
of the transition matrix used to find p^. The maximum likelihood estimate 
of p.j(t) is 



P..(t) = n..(t) / 2 



k= 1 



The numerator is the total number of students sampled who, given that they 
t- 1 t 

were in s. , are in s. , and the denominator is a normalizing factor. 
i J 

It can be seen that 

r 

2 



2 



P..(t) = 1. 

J = 1 J 

A A 

For large samples, the elements of P(t), P„(t), are multivariate 

normally distributed with mean p..(t), variance = 

2 r J 



. p ij (t > / 2 n y> 

j=i 



and covariance between 

Pjj(t) and p gh (t) = Pgh (t) / 

i=g and 0 otherwise. 



2 

j = l 



n. . 
ij 



if 
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These results hold, conditional on the sancple at time t-1. A 

further result which can be deduced from the above for large samples 

a a 

concerns the elements of the vector p^, p^(s^). By the laws of condi- 
tional probability, we can estimate 

2 

,) = 2 

j=i 

The distribution of p (s.), conditional on the sample at time t-1, is 

V 1 

normal with mean 



Pt (s i 



p j i (t) Pt-i (s j ) - 



2 r 

I 

j = l 



Ji 



p t-l (s j ) 



and variance 



2 {pji<‘) d-Pj^))/ 2 

j=i j=i 



An upper bound to the variance is 
2 r ' 



n ji ( t) 



L / \ 

p t -l (s j> 



1 / 



(‘I 



n ji (t) 



which is obtained by noting that the maximum of 
2 r 



2 ». 2 -. <*j> 



j = l 

2 

occurs when p, , (s.) 

t-1 j 

is equal to 1 for one value of j and zero for all others. 

Certain hypotheses also can be tested. The first of interest is that 
the transition matrices are all the same for differing values of t. Testing 
this hypothesis is equivalent to testing the Markov process for stationarity. 
The likelihood ratio test of size 1 - fljis useful here, and rejects the null- 
hypothesis that the Markov process is stationary if 



-2 log A 5 ^(o() 



■e ' (T-1) 2 r (2 r -l). 



The quantity A is defined as 
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I 



T 2 

A = 71 'IT 

t-1 i= 1 



r 



2 

7T 

j=i 



A 

P 



II 



p..(t) 



n. .(t) 
ij 



so that 



1 



-2 log e a = -2 



t=l i=l 



2 r 2 r 

2 2 

j = l 



n. .(t) 



ij 



log p.j - log p.j(t)| 



and / X(^)( T . 1 ) 2 r (2 r - 1 ) is the 100 ( 1 "" < ) 0 / 0 point of the 50 distribution with 

(T-1) 2 (2 -1) degrees of freedom. The quantity p.. can be calculated like 

A /4A -r 1J 

p..(t) if 
ij T 



n. . 



2 



(t). 



t=l 

Another useful test is that of the hypothesis that the estimated Markov 
transition matrix is equal, row by row, to a specified set of probabilities: 
P i;J .(t) = p° , all j = l, . . . , 2 r and i fixed. The chi -squared test is appropri 

here, for large samples, and the hypothesis of equality is rejected with 

2 



probability of Type I error if 
2 r 2r 



e 



- 7 7 

1 ' jtr k * 



(t> 



pjjW - 4 



pr 

ij 



is greater than the chi- squared 100^% point with 2-1 degrees of freedom. 
By convention, elements p?. = 0 are not permitted. 
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Note 2. Bayesian Estimation for Markov Chains 



In educational statistics, paucity of data always represents a problem. 

For example, if r subjects are considered, then each of the T transition 

2r 

matrices contain 2 elements, each of which must be estimated. This 
number is computed for a few values of r: 



• 4 



16 



64 



256 1024 4096 



Even if* for r =4, 256 observations were available, there is no 
guarantee that each element of the matrix will be estimable from the 
data, i. e. , v/ill contain at least one sample observation. 

Bayesian statistics allows the use of subjective evaluations to set 
all of the initial values of the elements of the transition matrices 
judgmentally. As data become available, they are used to modify the 
estimates of the elements of the transition matrix until, finally, the 
data totally determine the estimate and the effect of prior judgment 
vanishes. The following paragraph develops the required theory with 
some rigor and then the results and their use are stated. 

Assume that a set of random variables have an a priori m-variate 
Dirichlet distribution. This distribution is defined in Wilks (5). The 
joint prior probability distribution of the x i is, over the simplex O^x.^1, 

i“ 1 i , m, 



f(x. , . . . , x ) = 
I m 



m-fT 

2 

i= 1 



- 1 



(C 1' 1)! * c 2 - 1 * 1 * ** * c m+l 



c i - 1 

X. A . . .X 



m 

m & 



ri'i+1 

with the expectation of x. equal to c. / 

1 1 j=i 

equal to 



c., and the variance of x. 

J X 



c. (c, +. . . + c , . - C.) / 
i 1 m+1 r 






Higher order and mixed pnoments can also be determined. The parameters 

c A are assumed to be positive integers for our application of this distribution. 

It can also be shown that the sum z=x. +. . . + x has a Beta distribution 

i m 



f 



1 



I 



m 



^ with parameters e,,. .. +e m , j , and hence E(a) e 2s 

I 
[ 

I 



c. 

1 



m+1 

i 

i=l 



c. 

l 



If a sample of n is drawn; and y. falls into the j th out of m+1 categories, 
then the likelihood of the sample is 



L (y! > • • • > y^p , ... f x _ l ) - 



m 



m . h, j— 1 

~rr (y 4 )! J 

j=i 3 



£ x - yj 
" J 



n- 



m 

L- \ ' 



r^ii- 

d=l 



t m 



x, 



j nr . c.+y. _ ~ 

K ' 7 T 3 3 ■ 1 d-7 x> c y . 

. , J J- m+1 j = l 7 j-l 



conditional on the sample observed. By standard use of Bayes' theorem, the 
posterior distribution of the x. is proportional to the product of the likelihood 
and the prior distribution, or f'(x lf ... # x y, , . . . v ) = 

1 rrr 1 ’ J m 

(1- / x.) c . 

Z_j j- m+1 

j =1 j=i 

where K' is a constant not depending on x.. If the decision -theoretic loss 
in estimating x. is quadratic, i. e. if an increase in error causes an increase 
in the square of the loss, then the optimal (minimum expected loss) estimate 
of x. is the mean of the posterior distribution. Hence, x. is best estimated 

by (Oj + yj ) / r ^ 1 ' J 



[ 



m+i v 

2 V- • 

j=i / 



The work of the previous paragraph can be applied to estimating the 

rows of a Markov transition matrix as follows. Estimate the entry p..(t) 
subjectively as ^ 

c. ' 2r 



for all j = l, . . . , 2 r . • 

/ L c y 



j=i 

Take a sample of data to determine the number in state j at time t given 

that they were in state i at time t-1. After gathering the data, estimate the 
e ntry , _ 



t \ / 2r 2 r 

P ij (t) aS ( C j = "ij (t M // J V § n ij(‘) 

l 1=1 j=l J 



o 

ERIC 



m 



2 y j 

j = l 



I 
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Under the assumptions of the previous paragraph, the standard deviation of 

the new estimate of p..(t) is the square root of 

"J 




It can be seen that, as the sample size increases, the effect of the prior 

specification diminishes. Using these expressions, one can estimate 

parameters as data becomes available. The advantage of the Bayesian 

procedure is that a scarcity of data does not necessarily imply an estimated 

matrix with many zero elements. For small amounts of data, however, 

prior judgments are very important. The procedure described is 

conditional on the sample at time t-1. 

One problem remaining is the determination of the c., If the c. are 

all taken to be zero, then the adaptive estimate is equivalent to the 

maximum likelihood estimate described earlier. In the more general case, 

ft) 

the a priori estimates for the conditional probabilities p:. ; can be specified 

by an expert on education who has experience with the school in question. 

By asking the question, "To what size sample do you feel your judgment 

2 r 

is equivalent, " one can set the value of 



c., 

J 



Each of the individual c. 

J 



can be determined as c.: 

J 



>!!> 

u 



2 

2 

j=i 



j' 



C. 

J' 



While the adaptive estimation procedure here is useful in dealing with 

skimpy data, there is much statistical work to be done in validation of 

assumptions, choice of loss functions, etc. , in Bayesian analysis of 

0 

Markov chains. It is felt that this topic is a fruitful one for further 
research. 
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Note 3. Validation of the Markov Assumptions 



The assumptions of the Markov model were outlined earlier 
in detail. These assumptions must be validated before the model can 
be applied to actual data. For validation purposes, a population of 
more than two hundred low achievement children from a suburban New 
England school were studied. This population represented all of the 
approximately 2500 students in attendance in the high school who were 
poor performers and who had long records extending back through 
elementary school. 

For a given set of pass-fail states, the Markov assumption 
states that 



P t+1 = P t P t = P t P t-1 P t-1. 

2 

Writing the matrix with elements p(s. at t+1 (s. at t-1) as R^, we expect, 

J 

if the Markov property holds, 



P t+1 = R t-1 P t-1 = P t P t-1 P t-1, 



and 



Ki - p t p t-i 



Similarly, if the Markov property holds, 



R 



k 

t-1 




P t-l+j . 



By estimating the left and right hand sides of these relationships 
from data and comparing them, one can test the validity of the Markov 
assumption. It is expected that there is to be sampling fluctuation in 
the elements of both sides, so that any comparison and interpretation 
must be considered in light of the sampling variances. 



k 




% 
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Selecting multiple samples from a small finite population implies 
that there is overlap, i. e. , that some observations occur in more than 
one sample. If f is the sampling percentage, the ratio of the sam pling 
size to the population size, then 




is the expected proportion of people in j or fewer samples out of m 
samples drawn. A table of this number, for selected values of m and 
f, for j=l, is shown below. 



Expected % of Observations Common to One or Fewer 
of m samples, with each sample an 100f% sample 

0. 25 
0. 32 
0.50 
0.75 



3 4 m 5 7 10 







63 




23 






52 


22 




50 


31 


18 




1 






2 




m 

t 



It was decided to effect a trade-off between sampling variance 
due to small samples, and high correlation between samples by choosing 
f a 0*40, a forty per cent sample. Due to missing data, etc, , the actual 
sampling percentage used was one or two percent smaller. 

From the characteristics of the data available, it was decided 
further to stratify the data by sex and by drop-out vs. graduate'. A set 




% 



l 
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* « ' # m* 1 *??.** m i & s ips^rj 



of sampling experiments or comparisons were carried out to check the 
Markov hypotheses. These comparisons were: 



P 

4*4 


compared with 


P 6 R 


P 6 P 5 


compared with 


R 4 


P 5 P 4 


compared with 


R 3 






The first comparison measures two matrices describing transi- 
tions from the third grade to the sixth grade. The first of the two 
matrices is the product of the matrix describing direct, conditional 
probabilities of sixth grade states, conditional on fourth grade states, 
and the transition matrix from third to fourth, compared with the 
direct transition matrix from third to fourth, compared with the direct 
transition from third to fifth, multiplied by the transition matrix from 
fifth grade to sixth. The other comparisons follow analogously. Grades 
three through six were chosen because the data for those grades was 
most complete. 

There are five distinct matrices in the three above comparisons. 
Each was estimated from a separate 40% sample, and the samples were 
all matched for the stratifying categories; i. e. , each sample contained 
40% of the number in each of the four population subcategories. 

The standard errors of the elements of the matrices, obtained 
from a small finite population, explain some of the observed variation 
between the two sets of matrices. .If the Markov property is true, 
and if both sides of the above are post- multiplied by the p . vector, 

L"* JL 

where t corresponds to the subscript on the P matrices on the left- 
hand side, and t-1 corresponds to the subscript on the R matrices of 
the right-hand side, the final state vectors can be compared. In other 
words. 
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r »n» W «w.' 



1 



R 4 P 4 p 3 ~ p 6 


compared with 


p 6 P 6 R 3 p 3 


P 6 P 5 p 4 = p 6 


compared with 


P 6 = R 4 p 4 


P 5 P 4 p 3 = p 5 


compared with 


p 5 = R ^ ?3 



and only state-vector s need be compared. Tests such as the X 2 test and 
t-tests generally used to make paired comparisons of this sort are only 
approximately valid here. The correlation between state vectors cal- 
culated by this procedure has not been worked out, although it most 

probably, judging by the above table of expected overlap between samples, 
is positive. 

The first set of analyses considered four subjects, mathematics (M), 
spelling (P), writing (W), and reading (JR), each with a pass-fail threshold. 
This analysis considers, therefore, 16 states. Inclusion of one of the 
letters implies pass, exclusion implies fail; NON denotes failure of all 
subjects. The three comparisons noted above are presented in Analyses 
1-3. It should be remembered that the base vector is based on the actual 
records available for the population in grade 6, but the computed vectors 
are based on the total population in grade 3, a number generally smaller. 
This explains the disparity of the total of the base vector vs. that of 
the computed vectors. In order to compare the computed vectors with 
the base vector, the quantities shown must be normalized by dividing 
by the totals shown under the headings. Small disparities between the 
totals of the base vector columns for different analyses, and between 
the computed vectors are due to round off to preserve integer values. 

A procedure to reduce these disparities has been implemented in later 
analyses. 

It can be seen from these tables that the two most heavily occupied 
states, the states of passing everything and failing everything, are almost 
identical for the two computed vectors, indicating that the Markov 
property does hold for the four subjects considered over grades 3-6. 
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Even though there is less of a percentage agreement between some of 
the other states as computed by the two procedures, it appears as 
though the Markov model provides a reasonable approximation to the 
situation studied. 

As was pointed out earlier, the Markov property does not 
necessarily hold for a combination or elimination of certain states. 
Two studies were made to study this property, one combining the 
states writing and spelling into a new state (E), the other deleting 
spelling entirely. The comparison of the computed vectors is shown 
in Analyses 4 and 5 for the first comparison only in each case. It 
can be seen from these tables that, while the Markov property seems 
to hold as before, the agreement between the computed vectors is not 
quite as great as before. Care must indeed be taken in collapsing 
or deleting states in a Markov chain. 

A conservative upper bound for the standard deviation of the 
difference between elements of the two computed vectors is 



S. E. (computed \ector ^element. - computed vector ^element.) 




2 • TOTAL in smaller of computed vector columns 



which can be derived as follows. 

If m,. and m 9 . are the elements in question, then p,. = il 

11 .cl ^ *Ti Ton 



m~. i 

and p 9 . =_ th. have variance — 

TOTAL 4 TOTAL 

the total in the smaller of the two columns. 



m. . 
_LL_ 
TOTAL 

, where TOTAL is 



Hence, var(m^. - m^,^) 
“TOTAL 



2 cov (m H , m 2 . ) 



2 TOTAL 



li 1 
TOTAL 



The correlation between samples implies that the covariance is positive 



I 

I 






and hence 1/2 TOTAL is an upper bound to the variance. Because 
TOTALS 200, the bound for the standard error is about . 05 for 
each element of the difference between the computed vectors. 
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ANALYSIS NUMBER 1 

..COMPARISON OF STATE VECTORS FDR GRADE 6 

(Writing and Spelling Combined into E) 

BASE. VECTOR GRADE 3 .X PROBABILITY OF TR ANSI TI ON.. GRADE .. 

X PROBABILITY OF TRANSITION GRADE 



3. ...TO. 4.. 

4 TO 6 



BASE. VECTOR GRADE .. 3.. X .PROBABILITY OF...TRAN SI.TION ..GRADE._3.~T.O ... 5. 

X PROBABILITY OF TRANSITION GRADE 5 TO 6 
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CHAPTER VII 



THE DROPOUT /TRUANCY SUBMODEL 



In the previous chapter, a model for forecasting changes in achieve- 
ment due to Title I programs was discussed. In this chapter we measure 
changes in the dropout and truancy rates due to changes effected by a 
Title I program. For example, if a Title I program applied to the 
second grade causes an increase in student achievement at the end of 
the ninth grade, this projected change in achievement can be used to 
predict changes in the truancy and dropout rates during the tenth grade. 
Changes in dropout and truancy rates are calculated from changes in 
student achievement and student impedance to learning. Community 
factors also influence these rates, but are not affected, at least in the 

short run, by Title I. 

The effects of a Title I program applied in a given year (immediate 
effects), as measured by their relevance to and impact on the students 
involved (see Chapter IV), can be transformed into projected changes in 
achievement after the Title I program has been applied. The projection 
of these changes is discussed in Chapter V. Not only does achievement 
change, however, but students' motivation for learning also changes. 

This change in attitude can be measured in part by the change in the 
number of dropouts and truants in the years following the application of 
a Title I program, which we relate directly to the projected change in 

achievement. 

The projection of changes in dropout and truancy ratio is useful 
not only for its own sake, but also for projecting changes in educational 
opportunity and projected earnings in future years; these last two topics 

are discussed in more detail in Chapter IX. 

The change in the number of dropouts and truants during a given 
grade is projected by a linear function of the change in achievement m 
that grade due to a Title I program (dropouts normally occur only in 
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grades nine through twelve, due to the enforcement of compulsory education 
laws). The relationship of changes in dropouts and truants to changes in 
achievement is computed, based on a formula which uses evaluations by 
educators as well as available data. 

Dropout and truancy rates for a given year can be computed either 
as a fraction of the students who began the given year, or as a fraction of 
students present at a specified fixed time in the past. These ratios will 
differ because of changes in the school flow population due to dropouts 
during students' ongoing school careers, or transfers and deaths. The 
first rate can be interpreted as the conditional probability of the average 
student's becoming a dropout or truant during a given grade, given that he 
began that grade; this definition is analogous to the definition of rate of 
mortality used in life insurance tables. The second rate measures the 
probability that the average student becomes a dropout or truant in a 
certain grade, given that he entered the school population at a certain time 
in the past, such as first grade. Changes in these rates due to changes in 
the number of dropouts and truants, as a function of a Title I program, 
are computed directly. 

There are many factors which affect the dropout and truancy rates; 
these factors can generally be categorized as community factors, im- 
pedance factors (see Chapter V) and achievement factors. The community 
factors are measured by variables which describe the socio-economic 
environment in which students live; the second are described in terms of 
the home environment and children's attitudes toward the classroom and 
formal instruction, and the third are based on students' grade averages. 

A more specific delineation of the various factors is given after a discussion 
of the legal definitions of truancy and dropout and of some of the school 
system errors in data reporting. 

Although the legal definitions of truancy and dropout vary from dis- 
trict to district, there are certain general principles which are consistent. 
Truancy can be considered as the absence from school of a child of school 
age without parental knowledge, Unverifiable medical excuses and absences 
to care for sick relatives seem to comprise a large portion of these unlaw- 
ful absences. Legally, unlawful detention at home of children by parents 
does not constitute truancy, but the absolute numbers of unlawful absences 
and truants are so small that we make no distinction in this model. 
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Dropouts legally cannot occur before a child is eligible for working 
papers. Although students may be absent for extended periods before 
that time, they are not generally considered dropouts. The local Board 
of Attendance probably classifies these children as truants, and is 
responsible for investigating the causes of absence. After a student 
passes the age at which dropping out is legal, truancy rates may no longer 
be meaningful, and extended absences can be considered instances of 
dropping out. 

The measurement of truancy and dropout may reflect certain 
systematic inaccuracies. Classroom attendance reporting or child 
accounting often plays an important role in a school's finding. For 
example, New York City schools used to receive support as a function of 
average daily attendance. These schools tended to remove students from 
the rolls who were absent, legally or illegally, and to re-enroll them upon 
their return. Absence rates therefore tended to be biased downwards. 

The large student case load pressures on Boards of Attendance is fre- 
quently further compounded by their obligation to verify children's 
absences and to determine their causes. Large case loads may introduce 
further inaccuracies into the reporting of absences and their causes. 

Dropouts and truancy traditionally seem to be closely related to 
similar causal factors. Although these causal factors are impossible 
to measure on a large scale, quantitative surrogates can be found to 
replace them. As stated earlier, some of the causal factors commonly 
used to explain truancy and dropout rates relate to the classroom environ- 
ment and the child's ability to participate in it, the community attitude 
towards education, parental background and the positive or negative impetus 
it provides for a good education, the quality of the education provided, 
the season of the year, and the student's age. 

There is a quantifiable proxy variable for each of these factors. The 
classroom environment can be measured, at least in a rudimentary way, 
by the amount of space and material available for each student and by the 
number of students in a class. The monthly rental rate in the neighborhood 
provides a rough guide to the economic position of the community, and, 
even more roughly, a measure of its acceptance of the need for education. 
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Parental education, income levels, and presence /absence of the 
father in the home provide reasonable surrogates to define the home 
environment for individual students. In aggregate models, community 
averages can be used. 

The quality of education varies from class to class within a school. 
This variation is reflected in an absence rate which varies markedly with 
class schedule within a given school. Truancy and dropout are functions 
of time, as well. As male students approach the age of sixteen, when 
generally they can legally obtain working papers, their truancy and 
dropout rates increase. Female students tend to be absent earlier, due 
more perhaps to parental pressure than to educational causes. 

Other factors that determine attendance rates include the physical 
and mental health of a student, and the "social pathology" or abberations 
in the homelife. 

A more extensive study of truancy can be found in a study prepared 
in 1949 by the Citizens Committee on Children of New York City, Inc. 

Much of the previous discussion is summarized from findings presented 
in that study. An attempt to predict dropout rate as a function of community 
characteristics is given by R. Dentler and M. Warschauer (1965). 

Numbers of dropouts and truants for a given grade and type of student 
depend, as we have seen, on community factors, the child*s impedance to 
learning, and his achievement level. Title I programs do not have an 
immediate effect on the community environment of the students, so that a 
change in the number of dropouts or truants due to Title I can be considered 
approximately independent of community factors. The application of a 
Title I program in, for example, grade 3, may change the average student‘s 
impendance to learning and his achievement. These changes will continue 
and be propagated to some extent throughout the students educational 
career, in a way which can be measured by the projected change in student 
achievement for the years following application of a Title I program. A 
relationship between projected changes in the number of dropouts and 
truants after Title I application and changes in impedance and achievement 
at the time of the Title I program can therefore be studied in terms of 
the projected change in achievement following Title I. 
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It should be noted that measurement of impedance is a difficult task, 
and projecting impedance is even more difficult. Achievement data, 
while certainly not representing all information possible available for 
predicting changes in truancy and dropout, are available in almost all 
schools and are familiar to educators. The lack of availability of other 
data, more than theory, restricts us to the use of achievement data for 
predicting changes in truancy and dropout rates. Similar reasoning 
leads to the use of a linear model, rather than one with higher order 
terms and more parameters. If it is found that nonlinear relationships 
exist and can be well approximated by linear ones over certain regions, 
then the procedure described here for predicting changes in dropout and 
truancy rates can be applied. 

In view of the previous paragraph, we define linear relationships 
between the projected change in achievement at grade k,, written A A k _ x , 
and the change in the number of truants and dropouts at grade k, written 
A n£ and An k , respectively: 

A n k ^ bj A A k-1 , 

and AA k l . 

The quantity A A k _ j , was defined in the previous chapter as the projected 
expected achievement at grade k-1 following the application of a Title I 

program less the expected achievement at grade k-1 if no Title I program 
had been implemented. 

The constants bj and b^ could be determined by the least squares 
regression procedure if a sufficient number of observations were available 
from different schools in communities with similar socio-economic and 
demographic characteristics and of comparable educational level, on both 

•A n k ’ ^ n k * and A A k-1 ' Because Title I programs are relatively new, 
no data are or will be available for some time. Prior judgments must be 
used to set the values of bj and b 2 , and these judgments should be modified 
by the use of data as it becomes available. 

The parameter bj can be determined by asking the following two 
questions: 
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(a) Given the current level of achievement in grade k-1, what is 
your best guess of the number of students who will drop out if their 
achievement is as shown but who would not drop out if their achievement 
were to increase one grade level at this grade? 

(b) What is the range you would give around your answer in (a) so 
that you have 50% confidence that the actual correct number is included 

in the interval? (That is, your answer to (a) may not be precisely correct- 
what is the range in which you are 50% certain the real number will fall?) 

If the answer to question (a) is written Bjj, and the answer to (b) 
is written B^* then a good estimate for b^ is 

*1 = - B n : 

the standard deviation which the estimate is imputed to have is 36, 

as shown by Mosteller (see references at the end of the chapter). As 
data become available, a new estimate should be calculated which utilizes 
the new data. For calculation of such an estimate, formulae are available 
in Pratt, et al. Calculations for b ^ follow' in the same way. 

The dropout rate during grade k can be computed, conditional on the 
number starting grade k, as one minus the number who enter grade k + 1, 
less population transfer through moves and deaths, over the nuinber who 
enter grade k. If is the number who enter grade k, and R^, is the 
number of net transfers and deaths during grade k (generally zero or very 
close to zero), then 



d k =1 



Mfc_i - _ Mk ~ M^-fl 



Ri 



M, 



M, 



+ 



M, 



For our purposes, R^ probably can be ignored. 

An alternative definition of dropout rate during grade k is one minus 
the ratio of the number starting grade k + 1 to the number starting the 
grade at which the Title I program was applied. This definition is useful 
again if the net immigration, emigration and deaths are very small. If 
the grade at which the Title I program is applied is taken to be grade zero, 
then 



D k =1 ~ 



M k+1 ~ R k _ M 0 ~ M k+1 



M, 



M 0 



+ Rr 
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The two definitions are related by the equation 



d k = 



D k 
TZ=T 



1 - Z D j 

j=i 



which is easily verified by direct substitution. 



A change in achievement of i* 1 grade k- 1 produces a change 

in the number of dropouts equal to bj & Aj^^ during grade k, assuming 
negligible immigration, etc. The change in the number of students 
entering grade k is 



+ 4 - t “k_ 2 • 

or, by our previous assumptions, 




b l(k-l) &A k-2 b l(k-2) ^ A k-3 “ b l /J A 0 ' 

This number can be used as the numerator in calculating D k ; because the 
bj’s for each grade are each negative, a positive change in achievement 
implies a decrease in the number of dropouts. The change in the uncon- 
ditional dropout rate during grade lc due to Title I program effects is 



^ D k~l 



+ 



k-1 

£ 





A A: 



M 



0 



-1 



The change in the conditional dropout rate can be computed very approxi- 
mately as 



A d 



k-1 



4. K & A k-l 

+ D l(k-1) 



M 



k-1 



if the change in the number of dropouts is small. 

For example, suppose that the following data is available from school 
records : 
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Grade 



9 



10 



11 



12 



Number entering 
grade 



Average 

Achievement 



Unconditional 
Dropout Rate 

Conditional 
Dropout Rate 



100 98 

7.4 7.2 

. 02 
. 02 



96 94 

7.2 7.0 

.04 .06 

.02 .02 



A Title I program is applied at grade 8, so that, in the following 
years , 



Average Achievement 

Af ter Title I 7.8 7.6 7.4 7.4 

It is estimated that bj = 0. 5 for all grades. Using the formulae just out- 
lined, we compute 



Grade 




10 11 12 



Change in Number 

Number Entering Grade -- +2 +2 + 1 

= - Change in Dropouts 



Change in Unconditional 
Dropout Rate 



- . 02 - • 02 - . 01 



Change in Conditional 
Dropout Rate 



-.02 ' -.02 -.011 



Truancy calculations can be made in analogous ways using b ? instead of b. . 
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CHAPTER VIII 



THE COURSE OF STUDY SELECTION SUBMODEL 



In many high schools, students have several choices of academic 
programs available: College, business, vocational, etc. In this chapter 
we define a method for determining the change in the proportion of stu- 
dents eligible for each of these programs due to an improvement in 
their academic achievement caused by a Title I program. 

It is assumed that the change in the proportion of students of 
a given student type who choose a given course of study can be determined 
from the characteristics of the achievement distribution of all students 
of that type. The effect of a Title I program on student achievement in 
future years can be projected from details of its initial impact on the 
students (see Chapter VII). The projected achievement distribution may 
be different from the distribution before the application of a Title I 
program, and the difference can then be translated into a change in the 
proportions of students eligible for each course of study. Radical 
shifts in the achievement distribution might imply that more students 
can shift into a particular course of study than the school's facilities 
might admit. To forestall this possibility, constraints are placed on 
the number of students allowed to enroll in each course of study, and 
a constrained allocation is made. 

To begin the analysis, the available courses*.^ study are ranked 
in terms of the achievement levels of the students vho are enrolled in 
them. A plausible ranking might be college preparatory, business, and 
vocational, in decreasing order of achievement. Using this ranking, 
historical student choice patterns, and the achievement distribution for 
students of a given type, we calculate achievement thresholds which 
categorize students in terms of course of study. After the Title I pro- 
gram has been applied, the achievement distribution for the given type 
of student is projected- -using the results of Chapter V--to the grade 
at which course of study selection occurs. The new achievement dis- 



ti lbution is then used, with the original thresholds, to determine the new 
proportion of students eligible for each course of study. Finally, the 
proportions are checked against the constraints determined by the 

school and adjustments are made until the proportions satisfy these con- 
straints. 

In Chapter V, the expected achievement and standard deviation 
of achievement for the average student were computed. We assume that 
achievement of all students of a given type, e. g. type defined by economic 
level and/or race, is distributed according to a normal (or bell- shaped 
distribution. The average (or mean) achievement level and the standard 
deviation completely specify the characteristics of the bell- shaped 
curve. 

Schools which offer course of study selection generally allow 
only one final choice, often to be made as the student enters the ninth 
grade. Altnough a student's choice of electives in earlier years may 
point towards a particular course of study, his final selection of a given 
course of study is heavily dependent on his achievement, relative to the 
average, at the time the choice is made. Students with achievement far 
above average tend to elect an academic or college preparatory program; 
students whose achievement is less than average often select a voca- 
tional program. Although this generalization will not hold for each indi- 
vidual student, it appears to be true in the aggregate. We therefore 
assume that the courses of study which are available can be ranked, 
so that the first is taken primarily by the highest achievement group and 
so on. Inherent in this ranking is the notion that there are achievement 
thresholds which determine the various courses of study. If a particular 
student has an achievement score between two given thresholds, he is 
assumed to be eligible for the corresponding course of study. This con- 
cept is illustrated by the example in Figure 1. 




Figure 1 



Again, this assumption should be approximately valid for aggregated 
analysis of student achievement. 

In the example shown, the fraction of students whose achieve- 
ment is greater than the academic threshold is assumed to choose the 
academic course of study; those whose achievement falls between the 
business threshold and the academic threshold are assumed to choose 
the business course of study, and so on. 

The thresholds can be computed in a sequential manner. If 
the proportion of students of a given type enrolled in the academic course 
of study is taken to be the area under the achievement distribution curve 
to the right of the academic threshold, then the academic threshold can 
be determined by relating this fraction to the parameters' of the distri- 
bution. One can then equate the area under the curve between the busi- 
ness threshold and the (known) academic threshold with the proportion 
of students of the given type enrolled in the business course, and solve 
for the business threshold. The process continues until all thresholds 
have' been determined. There is always one fevrerthreshold to determine 
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than there are courses of study. 

After the Title I program has been applied, the student achieve- 
ment distribution will shift if the program is effective. In Chapter V, 



computed. It is assumed that the thresholds defining course of study 
selection do not change even though the achievement distribution may 
change. T.his assumption is generally valid if the achievement changes 
ai c small, 'lhe thresholds may themselves represent educational policy 
decisions of a state or city education board, and hence would be un- 
affected in the short run by changes in the achievement distribution 
within a given school. The shift in achievement distribution is shown 
in Figure 2. It should be noted that, because the thresholds differ 
from student type to student type, equal changes in the achievement dis- 
tributions of two student types will lead to different changes in course 
of study selection, reflecting the differing attitudes and values of the 
student types. 




Frequency 



i 




Achievement After Title I 



Achievement 



Frequency 




Achievement Before Title I 



Achievement 



Vocational 

Threshold 



Business 

Threshold 



Academic 

Threshold 



Figure 2 
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The proportion of students in each course of study after applica- 
tion of a Title I program is equal to the area under the new achievement 
distribution between the two relevant thresholds which were calculated, 
from the old achievement distribution. If the Title I program does not 
affect achievement, the two achievement curves are identical, and the 
proportion of students in each course of study does not change. 

After the new proportions have been computed for each course 
of study, they must be compared with the constraints on enrollment in 
each course of study set by the school. The reasons for including con- 
straints follow. Changes in achievement distributions for certain stu- 
dent types may suggest a large change in the enrollment in certain 
courses of study; a trade school with a few college preparatory students 
might suddenly seem to be a college preparatory school. To attribute 
a change of this kind to a Title I program is unrealistic; an explicit 
change in school policy is implied, which is something not considered 
in the present analysis. The constraints on enrollment in particular 
courses of study may be caused by school policy or by the scarcity of 
physical resources, such as typewriters, shop equipment, or teachers. 

If a school has effectively no such constraints, then such constraints 
can be ignored in the analysis; if they are used, then the adjustment of 
proportions to justify the constraints is carried out for each course of 
study in decreasing order of achievement threshold. 

It is a straightforward matter to quantify this projection method. 
We write A^ and as the expected achievement and standard deviation 
of achievement before Title I programs for students of a given type in 
grade k, at which course of study selection is assumed to occur* These 
achievement measures, after the Title I program has been applied, are 
written A’^ and The numbers of students in each course of study 

before the Title I program are written n^ n 2 , . . . , rt , where there are 
p courses of study. We wish to find the proportions n',, n J 0 , . . . , n* 

12 p 

• choosing each course of study after the Title I program. The school 
sets constraints of m,, m^* . . . , m^ students as the maximum per- 
mitted in each course of study. The total number of students of a given 



• • • 



type is N, equal to the sum n. + + . . . + n^, which is also equal to 

the sum n'^ + n'^ + . . . + n* and less than or equal to the sum. 

m l * m 2 * • • • + m p» 

It is assumed that the courses of study are ranked so that the 
first course of study corresponds to students with highest achievement, 
the second to the next highest achieving students, and so on. The func- 
tion R(u) is defined from the tables of the unit normal distribution as 
the area under the curve to the right of u. This function is illustrated 
by the shaded area in Figure 3. 




It is assumed that the proportion of students in each coiirse of study 
before Title I program application has been adjusted so that none of 
the proportions violates the constraints imposed by the schools; that 
is, for all courses of study, n is less than or equal to m/ 

The course of study thresholds are written B^, , B . (there 

is always onefowr threshold than course of study) and are calculated by 



R f(B - A k )/S fc 7= n 2/ N 

R tfB 2 - A k )/ Sk 7 - R ZlBj - A^/SJ = n 2/ N 



* 
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and so on, until the last: 



1 - R * B p-l - W = "p/n 



The last equation serves as a check, because n^ + n^ + . • • 

Tables of the unit normal distribution and an explanation of their use 
to compute R(u) are given at the end of this chapter. 

The new numbers of students enrolled in each course of study 
before considering the constraints is given by 



n'j = NR f(Bj - Ay/S'^7 

n' 2 = N (R/’(B 2 - Ay/S'j^? - R tfBj - A' k )/S‘J) 



and so on, until the last 

n ' p = N d- R 



The constraints are utilized as follows. Define min (a, b) to 
be the smaller of the two numbers a and b. Define also a surplus 
variable T^ which represents the number of students who are 

unable to be assigned to courses of study 1, 2, . . i, because of the 
constraints. Then the constrained proportions of students in each 
course of study after Title I, written n.* , ...» n *, are given by 

A r 



= min (n'p m^) 

= min (n^ + Tp m 2 ) 
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and so on to 



n 



min (n 1 + T ,. m ). 

P p-1 p 



where 



T. , = T. + n'. _ - m. , , 
x+1 x l+l x+1 



i = 0 , . . . , p-1 and T q = 0 



As a check, T = 0 if the allocation has been performed properly. The 

P 

"slack" variable T represents the surplus or deficit of students yet to 



be assigned. 

For example, suppose that students (all of the same type) in 
grade 10 have an expected achievement of 9. 4, in grade level units, 
and a standard deviation of 1. 0. There are three courses of study: 
college prep, business, and vocational. Out of the 100 students, 20 
choose the college prep program, 30 the business program, and 50 
the vocational program. Hence, = 9. 4, S k = 1. 0, n^ = 20, n 2 = 30, 
n 3 = an< ^ ^ = The school sets constraints of 28 in the college 

prep program, 30 in the business program and 42 in the vocational 
program, so that m JL = 28, m 2 = 30, and m 3 = 42. 

The achievement thresholds and B 2 can be computed from 
normal distribution tables as 



R f^o Li )= - 2 - or Bl ~ 9 ~ 

\ 1. U / 1.0 



= . 85, or B x = 10. 25 



R 



— ) - . 2 = . 3, or B 9 - 9. 4)/l. 0=0, or B 9 = 9. 40 

1. 0 2 2 
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After the Title I program has been applied, the average achievement 
is assumed to have increased to 9. 7, but the standard deviation re- 
mains unchanged. The new proportions in each course of study, tempor- 
arily ignoring the constraints are given by 



n. 



n. 



n. 



1 



100 -- = 29 



= 100 - R ? f ~^)] = 100 ( 62 - . 29 = 

= 100 *jl = 100 (1 - 62) = 38 



33 



= min (29, 28) = 28 



T = 0 
o 

T x = 0 + 29 - 28 = 1 



n^r = min (33 + 1, 30) = 30 



T 2 = 1 + 33 - 30 = 4 



= min (38 + 4, 42) = 42 



T 3 " 4 + 38 = 42 = 0 
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CHAPTER IX 

COMMUNITY EFFECTS SUBMODEL 

The discussion thus far has centered on the projection of effects 
which are directly a part of the actual school environment. The 
Community Effects Submodel, with which this chapter is concerned, 
is an attempt to assess the impact of these immediately school- 
based effects from proposed Title I projects in the larger context 
of the community in which the target school population is located. 
Community effects are an important means by which to evaluate the 
educational system, for one important measure of the effectiveness 
is the long-term performance of the people who are its product. 

Two indicators of school-community interaction have been developed. 
These are: 1) potential lifetime earnings for the various student 

types as an indication of the economic consequences of education, 
and 2) an index of equality of educational opportunity for these groups, 
which indicates the degree to which achievement is associated with socio- 
economic background. 

Potential Lifetime Earnings 

The potential earnings subroutine is based on three factors: 
the grade level at which students drop out of school, the achievement 
level of those students who graduate from high school, and the courses 
of study chosen by students who attend schools in which several courses 
of study are available. These variables are assumed to have a strong 
effect upon the careers eventually chosen by the students and thus upon 
their earnings later in life. 

The potential earnings portion of the model consists of a distribution 
routine which determines the expected fractions of the original group 
of students which will fall into each future earnings category. As 
shown, in Figure 1, it receives inputs describing the achievement of 
graduating students from the School Flow Submodel, inputs describing 
the number and type of students choosing the various available courses 
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of study from the Course of Study Selection Submodel, and Inputs 
about the timing of dropouts from the Dropout/ Truancy Submodel. 

A number of studies indicate that there is a strong relationship 
between such variables as age of leaving school, achievement level 
of graduates, course of study (and thus to some extent, occupation) 
chosen by students, and their expected lifetime earnings or expected 
average salary during their working years. Reports published by 
the Bureau of the Census and various journal articles indicate that 
lifetime earnings are related positively to number of years in school, 
somewhat positively with high school achievement level, and varyingly 
with occupation (or course of study selected). The model used to 
calculate potential lifetime earnings incorporates these general 
relationships, but does not go into the fine detail discussed in some 
of these articles. 

The possibility of discounting the stream of future earnings to 
determine the present value of future income, of computing the return 
on present investment of educational programs, or of including growth 
rates for the economy as a whole and for different occupations as a 
function of differential demand in computing potential earnings were 
considered, but were set aside in the interests of a more intensive 
study of the educational process which is the core of the model. 

Index of Equality of Educational Opportunity 

The index of equality of educational opportunity is a measure which 
was devised to indicate the degree to which a school system exercises 
and develops the potential of all students regardless of socio-economic 
background. The idea for such an index comes from an article by 
James S. Coleman, who suggested that the measure of equality of 
educational opportunity is the degree to which each student is equipped 
at the end of school to compete on an equal basis with others, whatever 
his social origins. Another way of putting this is to say that the schools 
are successful only insofar as they reduce the dependence of a student’s 
opportunities upon his social origins - equality of educational opportunity 




POTENTIAL LIFETIME EARNINGS SUBROUTINE 




implies schools whose influences will overcome the differences in 
starting point of children from different social groups.* 

In order to compute an index of equality of educational opportunity, 
it is important to look at the change in achievement differences among 
student types during their school career. To the extent that a proposed 
Title J program lessens these achievement differences by the time 
of graduation, to that extent it contributes to increased equality of 
educational opportunity. 

The measure of equality of educational opportunity which we have 
devised to implement Coleman’s concept is computed as follows. 
Achievement distributions of first grade students will be available to 
the model as part of the input data. We assume that inter-group 
achievement level differences at this point arc due primarily to 
differences in the respective groups' home environments and, therefore, 
we take these differences to be our baseline, i. e. , to represent the 
initial inequalities which the school system seeks to eliminate. Using 
the procedures described in Chapters V and VI, the model computes 
the effect of a proposed Title I program on achievement distributions 
at graduation. If by the time of graduation these differences have 
increased, the school has contributed to the pre-existing disparities 
among the student types, and the school has provided unequal educational 
opportunities to its students. In output terms, this situation would be 
represented by a low value for the index of equality educational 
opportunity. 

In more quantitative terms, the model performs the following 

calculations. Suppose that we write A., for the expected achievement 

at grade j for student type i. Then A.j - A^ represents the difference 

in achievement between two student types in the first grade. Similarly, 

A.. - A, . represents the difference in achievement between two student 
U 4 K J 

types in the j grade. A measure of the change in educational opportunity 
for student types i and k from the first to the j grade is 

James S. Coleman, ’’Equal Schools or Equal Students?", The Public 
Interest , 1966 , 4, 70-75. 
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The closer this index is to one, the more nearly equal is educational 
opportunity. 

The calculation of A.., etc,, is described in Chapter VI. If 

the expected achievements after application of a Title I program are 

denoted by X*.., etc., then 
3 i 3 
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represents the change in equality of educational opportunity due to a 
Title I program. If the program increases equality of education, the 
measure Ae. k (j) will be positive. 

The measures presented are given in relative terms. It may 
also be of interest to test hypothesis concerning the absolute measures 
of achievement differences, based on the assumption that the expected 
achievement for different student types at first grade is fixed. 

If (Ay - A^j ) is taken as the measure of inequity of achievement 
between student types i and k at grade j, then, by the assumptions of 
Chapter VI, the measure is normally distributed with mean A.j - A^ 
under the null hypothesis of no change in equality, and variance 

jcTy + - 2 Pikj Oij Cfkj J » where is the correlation 

between an achievement score for student type i at grade j and a 
score for student type k at grade j; n is the number of students on 
whom the computations of Ay is based. 

The projected achievement values after Title I can be tested 
to see if they differ significantly from the previous values by using 
tables of the normal distribution if n 2 30. Hence, 
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is the test statistic, and if z ~ -1.64, there has been a significant 
(at the 95% level) improvement in equality of educational opportunity 
at grade j, if zr.rl.64, a significant lessening in equality of educational 
opportunity has occurred. At -1, 64 Iz z !_•; 1 . 64, no significant change 
has occurred. 

It should be noted that the use of hypothesis testing should be 
for information and guidance, but not as an absolute decision proceduie, 
especially because the results of this section are approximate. The 
value for p- kj should be determined from historical data or by 
judgment; estimated values of /V which are smaller than tne 
true value gives conservative tests in that quantities which are in 
fact significant do not appear so. If many tests or comparisons are 
to be made, then the 5% level should be replaced by a significance 
level (e.g. , 5%) divided by the number of tests or comparisons to be 

made, so that the overall comparisons are valid. The threshold 1.64 
should be changed appropriately. 
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CHAPTER X 

THE EFFECTIVENESS OUTPUTS SUBMODEL 



Throughout all phases of the modeling effort, it has been our intent 
to design a system that will be of service to those responsible for the final 
evaluation of proposed Title I programs. It is not claimed .that the OECE 

model will solve all their problems, but rather that it can provide educa- 
tion decision-makers with additional information on which to base their 
decisions. Every attempt has been made to insure that such information 
will be both meaningful and useful. 

The great mathematician Norbert Weiner once remarked that the 
trouble with computers is exactly the same as the trouble with magic — 
they give us only what we ask for, not what we should have asked for. 

With this in mind, much of the work has been devoted to a careful and 
highly critical selection, among possible model outputs, of those which, 
in the estimation of both the Title I group at OE and the Abt Associates Inc. 
staff, seemed most useful. But simply selecting the most useful outputs is 
not enough. Equally important is the manner in whcih these outputs are 
presented. Data, when presented poorly, is of limited usefulness and often 
misleading. A computer with a high-speed printer is capable of creating 
more printed data in an hour than a man can read in a week — this is both 
its weakness and its strength. A computer can serve a few men poorly — 
by overloading them with half-interesting mountains of data — or it can 
serve many men well— by providing clear and concise summaries of the 
results of its complex computations. Which it will do depends on the way 
its output is presented. Ideally, the report generated by the computer must 
be presented and organized for maximum convenience to the planner. 

To achieve this convenience, the project group has selected, in 
consultation with the Title I group at the Office of Education, a two-part 
presentation of the model outputs: (1) a summary page, designed to 

give the planner a quick overview of the projected effects of a proposed 
Title I program, and (2) several pages of more detailed information, 



designed to give the planner a more specific breakdown of the information 
presented in the summary page. Since Title I project evaluators are in- 
terested in seeing the changes effected by proposed programs, both parts 
of the output presentation provide not only a projection of the results after 
the program under consideration has run its course, but also a point-for- 
point set of comparison data which describe the situation as it exists 
prior to the initiation of the Title I program. 

Often, the decision-maker will wish to compare two or more alterna- 
tive Title I projects for the same school district. This is readily accomplished 
by comparing the model’s projected results for each of the proposed alterna- 
tives. Notice that here the model provides a particularly useful service. 
Alternative proposals may differ over a wide range— they may be directed 
at different grades, different subjects, and different target groups. The 
more they differ, the more difficult it is for a planner to compare their 
relative merits. The strength of the OECE model is that it has been de- 
signed to accept and evaluate highly dissimilar proposals in commensurate 
terms, e.g. , achievement levels, etc. , but in order for this strength to be 
fully employed, it is necessary that the decision-maker understand intui- 
tively the internal functionings of the model. This is not to claim that 
decision-making on the basis of model projections of program results will 
suddenly become a simple process, but that the model can provide an im- 
portant part of the information upon which a final decision is made. Indeed, 
the most important part of the model is the person who uses it. The model 
will present reasonable predictions of the effects of proposed programs, 
but it is the user who must weigh these possible effects against each other 
on the basis of his expertise and come to a definite conclusion. This, we 
feel, is the proper relationship between man and model. 

As has been stated earlier, current knowledge of the relevant edu- 
cational, learning, and sociological processes is not sufficient to construct 
a model which can predict precisely the changes in a school system and 
community that might result from the introduction of Title I programs into 
the schools. However, using current knowledge, one is realizable which 
will make reasonable predictions which may be compared across a series 
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of proposed programs. This procedure avoids the potential error of re- 
garding the results as absolute predictions, minimizes the model biases, 
and provides relative measures of different programs' advantages and 

A 

disadvantages. 

The Effectiveness Outputs Submodel is the communications link 
between the actual evaluative submodels described in Chapters V through IX 
and the user. It receives the calculated outputs from the evaluative sub- 
models and prints them in the form of a report for the user. As men- 
tioned above, the report has two parts: (1) an overall summary (Figure 2) 

and (2) a more detailed breakdown of the summarized data. Part two is 
divided into a detailed section on achievements (Figure 2) and a detailed 
section on dropouts and attendance (Figure 3). Since primary emphasis 
is placed on the changes which result from the introduction of a Title I 
program, the output report for each project run includes a parallel set 
of baseline data which the model calculates from the input data character- 
izing the school district but excluding any proposed program data. Base- 
line data are indicated in the report by the heading: 

BEFORE PROJECT 
YEAR: 19xx 

The Title I projected results are indicated by the heading: 

AFTER PROJECT 
YEAR: 19xx 

Where appropriate, data in the summary and in the achievement section 
are broken down by the sociological characteristics of the students. As 
noted in the previous chapters, the student body is divided, for the pur- 
poses of the model, into four socio-economic groups. Data for the indi- 
vidual groups appears under the heading: 

STUDENT ' | 

TYPE l 
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where the types are: 



Type 1 — 



non -whites with annual family income less than 
or equal to a threshold T, currently set at $2000 



Type 2 — non -whites with annual family income greater 
than T 



Type 3 — whites with annual family income less than or 
equal to T 

Type 4 ~ whites with annual family income greater than T 
S ummary Page 

The overall summary (Figure 1) shows the effects of a Title I 
program, on the school and community. Average achievements, values 
for the index of impedance to instruction, number of dropouts, and 
number of graduates are listed under the heading: 



SCHOOL IMPACTS 



These are defined as follows: 

1. Average Achievement is the average of student achievement 
grade levels in all subjects. It is given at the target grade and 
at the twelfth grade. If the target group spans more than one 
grade, the average is given for the highest grade. (See 
Chapters Y and VI. ) 

2. Index of Impedance to Instruction is a measure of the 
incidence within a student type of certain characteristics 
thought to be detrimental to the students' receptivity to 
instruction. Index values range from one to ten. High values 
indicate that the characteristics of the student's environment 
would tend to place him at a disadvantage in the school learning 
situation, low values indicate fewer environmental disadvantages. 
(See Chapter VI. ) 

0 

3. Number of Dropouts is the total number of students from 
the target population expected to leave school before graduation. 
(See Chapter VII.) 

4. Number of Graduates is the total number of students from 
the target population expected to graduate. (See Chapter VII. ) 



Data indicating projected degree of equality of educational 

opportunity and expected average lifetime earnings appear under the 
heading: 

COMMUNITY IMPACTS 

Index of Equality of Educational Opportunity indicates the degree to 
which students' educational achievement is dependent upon their socio- 
economic background. (See Chapter IX. ) The index in the summary is 
scaled so that if the condition of perfect equality is met, i. e. , academic 
levels are not limited by socio-econoinic background, the index will be one; 
if complete inequality occurs, i. e. , achievement levels are completely 
determined by socio-economic factors, the index will be zero. 

Expected Average Lifetime Earnings are projected for each of 
the four student types and included on the summary page. 

A chievements 

In addition to the information in the overall summary, more detailed 
data is provided about academic achievement (Figure 2). Achievements 
are given m grade levels; e. g. , a student reading at the beginning fourth 
grade level has an achievement level of 4. 0 in reading regardless of the 
grade he is actually in. These levels are shown for the average student 
of each student type, for twelve grades, in up to six academic subjects. 

Dropouts and Attendance 

Information about the number of dropouts and the attendance rate 
is provided in a graphical format (Figure 3). These graphs give a twelve 
grade breakdown for the entire school population; i. e. , the individual 
figures about dropouts and attendance for each grade are not disaggregated 
by student type. The attendance percentage is based upon total student 

days, which equals the number of students multiplied by 'the number of 
school days. 
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CHAPTER XI 



RECOMMENDATIONS FOR FURTHER RESEARCH 



The design of the present cost-effectiveness model raised 
many new questions which require further investigation if the 
effectiveness of public education is to be improved. If current 
educational research were to address itself to the following areas 
of investigation, comprehensive evaluation techniques for educational 
change could be much improved. For convenience, the recommenda 
tions suggested by this study are divided into substantive and 
methodological research categories. 

Substantive Research should be conducted to determine: 

1. Operationally defined and measurable indicators 
of teaching effectiveness that require only commonly 
obtainable data; 

2. The relative contributions of home, peer influence, 
and school instruction to student achievement; 

3. The relative contributions of service and instruc- 
tional improvement projects to student achievement 
change for various absolute levels of service and 
instruction; 

4. The existence of a minimum standard in school 
service provisions and/or level of instruction which need 
to be met before significant improvement can be expected 
in student attitudes and achievement; 

5. The relative quantitative contribution of the actual 
variables in the school environment which contribute 
most heavily to student attitude and achievement change 
(through protocol models of classroom response patterns, 
small group research, and role model analysis); 
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6. The coefficient and parameter settings for the 
environmental influences on student attitude and achieve- 
ment change with enough accuracy to allow useful 
prediction; 

7. A generalized classification system for community 
types to minimize data needs (a number of communities 
of a given type would be able to use one data base); 

8. The differential effects of various teaching strategies 
on various student types; 

9. The impact of Title I projects on the attitudes and 
behaviors of the parents of affected students, and parent 
feedback effects on student attitudes and achievements; 

10. The impact of Title I projects on the social, cultural, 
economic, and political structures of the community and 
the community's feedback to the education system; 

11. The later effects of early academic failures and the 
later effects of early academic successes; 

12. Student interest and achievement sensitivity to the 
sequencing of subject matter; 

13. The impact the overall organizational character of a 
school has on the performances of various student types; 

14. The impact of multiple and inter-school programs on 
the performance of the students and on the attitudes of the 
administrators, and how changes in the latter in turn affect 
the former; 

15. The impact of the newly racially-integrated schools 
(Project METCO in Boston, Massachusetts, for example) 
on white and non-white student performances; 

16. Guidelines, based on the implications of the model, 

for the changing role and character of the urban education 
system; . - 
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17. An expanded community submodel to include demographi 
and occupational forecasts of job supply and demand, espe- 
cially in education; 

18. The relationship of later economic and social status 
' to particular school variables to help derive measures of 

value for school changes. 

Methodological Research should be ‘conducted to: 

1. Program the present model for computer operation; 

2. Test the model with real data drawn from several 
project histories; 

3. Compile a handbook for Title I proposers which would 
give guidelines by community type for the kinds of projects 
likely to yield the most benefit; 

4. Determine the availability and comparability of school 
and community data across school districts; 

5. Develo|5 improved data collection procedures to support 
the model effort; 

6. Explore alternative techniques for extrapolating the 
long-range effects of changes in achievement (the present 
School Flow Submodel is a single-stage Markov process); 

„ 7. Examine the possibilities of using the cost-effectiveness 

model in all phases of Title I administration from project 
proposal to approval and implementation. 
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A DIFFERENCE EQUATION .MODEL OF THE IMPACT OF 
1 ■ ■ * ■»"» — — 

SUPPLEMENTAL EDUCATION PROGRAMS; A SUMMARY 



Introduction 

We consider that the impact of a supplemental education pro- 
gram on the students of a school occurs during the year at which the 
program is applied, over the remainder of the school career of the 
students, and after schooling, as projected community impact. We 
further consider that an impact is two-fold: it modifies, over these 
three time periods, the average achievement level of particular student 
types or classes, and it alters the expected number of students in non- 
academic courses of study and the number who are dropouts and truants. 

Based on these general assumptions and some more specific 
ones, it is possible to construct a system of difference cqixations which 
can be used to simulate the impact of supplemental education programs 
on a particixlar school, relative to alternative programs. These difference 
equations are qualifications of existing educational and sociological 
theory, and require only a limited amount of historical data. 

2. Mathematical Representation of Students in a Given Grade . 

Students in a given grade are stratified by student type, e. g. , 
white with family annual income less than $2, 000, non- white with 
family income greater than $2, 000 per year, and by course of study, 
e. g. , vocational, commercial, academic. In schools where no course 
of study separation is permitted, this disaggregation is not considered. 
Students of a particular type in a particular course of study in a given 
grade are further classifiable by behavior, e. g. , dropout, truanc, 
death, etc. We denote the number of students of type i in course of 
study j in grade k by nj? and add a second subscript to indicate be- 
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havioral classification: nj^ truant * s ^ e num ber of students of type 

1 in course of study j in grade k who are truants. 

In a given grade, there are a fixed number, r, of subjects 
offered. For each subject a pass-fail or satisfactory-unsatisfactory 
threshold can be defined (the generalization to three or more classi- 
fications is restricted only computationally), and each student is given 
a 1 or 0 as he surpasses or does not surpass the threshold for that 
subject. If this assignment is made for each student of type i in 
course of study j, then a vector of r binary digits can be defined for 

each student. If there are r subjects, then it is clear that there are 
r 

2 possible vectors. Each vector corresponds to a state, e. g. , in 
the two subject case, pass English, fail math, and fail English, fail 
math are two of the four possible states. We denote the proportion 
of students of type i in coxrrse of study j in grade k in a state s 
(corresponding to the s th vector of i's and o's, s = 1, . . . , 2 r ) by 

P k ( S )* The vector of proportions with 2 elements defined by pJ J (s) 
over s is called the probability of state vector at time lc for the l ^ 
student type in the j course of study, and is written Py 1 . It is clear 
that £ 

I P k <*> = 



S=1 



A student of type i in the j^ 1 course of study in grade k has an 

academic record represented by his state vector s. In grade k + 1, he 

has a new (or possibly the same) state vector s'. The proportion of 

students of type i in. course of study j who, given that they had a state 

vector s in grade k, went on to have a state vector s' in grade k + 1 is 

written P k (s /s ). The matrix of all 2^ r proportions of students of 

type i in the j course of study who, given their state in grade k, went 

• * 

on to a state in grade k + 1, is written . The rows in the matrix 

ij 

P k correspond to the states at grade k, and the columns to the states 
at grade k + 1. 
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Because achievement is closely related to the state vectors 

• • 

11 

P-ii( s )> it is possible to compute from them an expected achievement 

• • • « 

index. These quantities are written and respectively. 

We assume that students in grade k can be characterized by 
their student types, courses of study, behavior, their probability of 
state vectors, and the transition matrix which is used to determine 
their stat vectors in grade k + 1. The difference equations describing 
the impact of supplemental programs on students determine how stu- 
dent distribution over these characteristics changes when the pro- 
gram is applied over the school life of the student and later in terms 
of their earning capacity and equality of educational opportunity. All 
of the difference equations depend on these characteristics. 

3 . Derivation of Baseline Conditions 

The probability of state vectors P^ and transition matrices 

ij ^ 

for k = 0, . . . , are assumed to be known or estimable from his- 
torical data. The estimation procedure is adaptive, based on Bayesian 
statistical methods, in that it combines data, as it becomes available, 
with initial subjective estimates for the proportions involved. The 
time k = 0 is defined as the grade at which the supplemental program 
is applied. 

The achievement index statistics at each grade are determined 
from the probability of state vectors as follows. For each state vector 
s with r binary elements, there is a number ft (s) defined as the number 
of unit elements of the vector. For the two subject example, pass 
English and fail math, tf(s)=l; failing both yields )$(s)=‘0. The ex- 
pected achievement at each grade for each student type in each 
course of study is determined recursively as 

2 r 

A k+1 = A k + P k+1 (S) * where Vi = 0 

s==l 

1 

t 

and the standard deviation of achievement in grade k + 1, conditional 



147 



on achievement in grade k for the i student type in the j course of 
study is 




The statistics (1) and (2) are predicated on the assumption that the number of 
subjects passed is a measure of achievement; the first two moments of 
that random variable can be calculated from the probability of state vector 
as shown. 

The number of students in the grade of type i in the j^ 1 course 
of study with a particular behavior pattern is assumed to be available from 
school records. The set of these members satisfies certain balance and 
continuity restrictions as follows: 



_ 1J , 1J = 

m k k, dropout k, net transfer \+l 



(3) 



.i 

n 



“k'-Mfc 



(4) 



ij , ij 

n k, active n k, active 



= m. 



ij 



( 5 ) 



i 

The first equation is a continuity restriction. The subscript 
,f net transfer" refers to a net of transfers into the school less deaths 
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and transfers out of the school. The variable comprises, as is 

shown in (5), the number of students of type i in course of study j in 

grade k who are enrolled in the school at the beginning of grade k; the 

subscript "active" has no subjective connotation. Dropouts and trans- 

tVi 

fers are assumed to occur at the end of the k grade. Equation (4) 
defines the total number of students in the k^ 1 grade. 

4 . Impact at Time of Application of the Program . 

4. 1 Change in Probability of State Vector 

It is assumed that every supplemental program can be defined 
in terms of environmental (school and community) and psycho- social 
variables and its relevance to them. A particular set of relevant 
variables measuring the quantity, quality, and duration of a program, 
its relevance to the school and community environments for which it 
is intended, and certain other characteristics has been defined. For 
each supplemental program, measures are calculated which determine 
whether or not the program has improved the quality of the education 
available to students affected, of the type in course of study j, in 
the target grade o, and whether or not it has reduced their impedance 
to learning. From these measures, the fractional improvement in in- 
struction in subject Lfor i th student type in course cf study j in the target 
grade can be determined as f/*, and the reduction of impedance, a 
positive number, can be determined for each student type and course 
of study as f 1 ** . • 

A particular state s can be expected to have a higher proportion 

0 

of students in it than before the program if the program is beneficial 

and if the state corresponds to passing a large number of subjects. If 
th 

the It component of s, corresponding to subject L, is unity, because it 
is assumed that the proportion of people passing the L subject does not 
decrease after a beneficial program, the contribution of that subject 



o 
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to the new proportion of students in state s is positive. If that com- 
ponent is negative, its contribution to the probability of state is 
negative. Similarly, if half or more subjects are passed in a state, 
the reduction in impedance is assumed to imply an increase in the 
proportion of students in that state. 

Specifically, the new probability of state s in the target grade 
for the 1 students type in the j^ 1 course of study is 



r o < s > 



P D ij (s) + V (-l) 1 - V 'L (s) a? f y + (-l) r / 2 -tf< s > a ij fij 

U Li L * Li 



( 6 ) 



L=1 



subject to the restriction that 



- P o< s > - Y, (-l) 1 " V L (s> aj j f l T j + (-l )^ 2 ’* 5 (s) ai j f‘j - l.p*i(s) 
L = 1 



L L 



(7) 



for r, the number of subjects, an even number. 

X,l_ 

The variable v (s) is defined as 1 if the L n component of s is unity, 

• • • • 

and zero if it is zero. The constants a^ and a 1 ^ are set a priori based on 

J_i s — •*- 

educational theory. Equation (7) is a restriction constraining the new 
probabilities of state to lie between zero and one. 

4. 2 Change in Achievement Statistics at the Target Grade 

Given the probability of state vector with elements defined by (6) 
and (7), one can determine the change in achievement statistics for the 
target population by using expressions like (1) and (2). 




2 r 



y ( r D ij (s> - (s))tf( s ), 

s=l 



( 8 ) 



o 
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and 




4. 3 Change in the Number of Dropouts in the Target Grade 

It has been shown in the sociological literature that dropouts are 
a function of environmental factors, student psychological factors, and 
achievement, for a particular student type and course of study. We write 



n " j , = f (c,z 

o , dropout ' 



A) 



( 10 ) 



as a mathematical expression of this qualitative result. It is assumed that 
the supplemental programs affect only the psycho- social factors, by reducing 
impedence, and achievement, as shown inl>4. 2. The new number of drop- 
outs in the target grade is given by 



n 



ij 



*-J- A 1 J ^ f (q 

, dropout Lin 0 , dropout ' 1 



z 



ij 



+ C J 

S 



1J 



+ 



Aa «) 



(id 



If we expand (11) in a one-term Taylor's series and subtract (10), the result is 



AJj - d± flj , 2i Aa'i 

o , dropout fiz ss $A' A o 



( 12 ) 



by assuming that the function f is almost linear in f and A in the region of 
interest, we have, approximately, 



An ij . . = f « +P i i AA ij 

o , dropout o s /-'o o 



(13) 
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subject to the restriction that the lefthand side of (11) is non-negative. 
The variables on the righthand side of (13) have been calculated in 4.1 
and 4.2; the parameters are set initially on the basis of educational 
theory, and modified adaptively using Bayesian regression methods as 
observations on (13) become available. 

4. 4 Change in the Number of Truants in the Target Grade 

Because data and sociological theory for truants are less well 
specified even than that for dropouts, it is difficult to construct an 
elaborate model. One can make an argument similar to that for drop- 
outs, and conclude that an approximate representation of the change in 
the number of truants of type i in the j course of study in the target 
grade is 




truant 




(14) 



subject to the non- negativity of n 1 ^ . # , + A 

o i tru&nc o j t 2 ?Ucint 

parameters are to be estimated adpatively. 



where the 



5. Yearly Impact Until t he Class Graduates 

5 . 1 Change in Probability of State Vectors and .Achievement Indices 
at Grade k+1 

The change in probability of state vectors is determined based on 
the assumptions of a Markov process, in which the initial state vector is 
modified repeatedly by a one- step transition matrix. The change in 
probability of state vector is determined recursively by 



,1J 



- s (r XJ « p lc=0 

1*1, )> K u * • • • 






k+1 k+1 ~k+l 1 k 






■ • • • * « 
where r ^ is determined in 4. 1 and pN . P^ are determined from 

O • ~KtI| K 

historical records. 



(15) 
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The new probability of state vectors can be used to calculate a 
new mean and standard deviation of achievement in grade k+1 for a 
particular student type and course of study. The equations for these 
computations arc identical v/ith (8) and (9), except that the zero sub- 
script is replaced by k+1. 

5. 2 Change in the Number of Dropouts and Truants 'at Grade k+1 

The dropout and truancy relationships of 1^4. 3 and 4.4 can be 

modified for use in grades following the application of the supplemental 

program. Both equations require the change in achievement in grade 

k+1, which is determined in 5.1. In the dropout and truant equations 

in b4. 3 and 4. 4, an impedance factor is also used. The change in im~ 

^ ii 

pedance f J is subsumed in the calculation of the change in achievement, 

D 

so that only one variable is needed to predict dropouts and truants in 
years following the application of the supplemental program: 



AnjJ. 



= n 






k+l , dropout 



k+1 



Aa 



k+1, 



n 1J = d l J A A 1 * 1 

n k+l, truant V k+1 K+1, 



( 16 ) 

U7) 



subject to the non -negativity of the new numbers of dropout and truants. 

5 . 3 Change in the Course of Study Distribution 

At one time during the school life a class, the students are required 
to choose a course of study with an associated curriculum plan. This choice 
is made by each student type, and may be made at or after the application 
of the supplemental education program. The course of study distribution 
is predicated on the assumption that students in the courses of study 
available can be ranked in decreasing order of expected achievement, 
i. e. that 



for all k . 



(18) 
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It is further assumed that the moments and 6j^ define a normal 
distribution, and that there is a threshold achievement for each course 
of study which is determined by a percentage point of the normal dis- 
tribution with moments defined by the achievement indices for a given 
student type and each course of study. For example, students with ex- 
pected achievement between the thresholds for the j^ 1 and (j-1; course 
of study arc assumed to be in the (j-lf^ 1 course of study. If the mo- 
ments of the normally distributed random variable representing stu- 
dent achievement for i at grade k are, for p courses of study, 

P 



4= 



< — 
Z 



in. 



P 

iL A iL IK 
k 



j-1 



*k IL, “k 

3=1 



iL 



(19) 



and 



P 






y 

j=i 




then after the application of the supplemental program, these moments 
change by 



P / P 

A _i V" iLt A U/V iL 

^ S k = L, "V-^k L “k 

3=1 / 3=1 



( 20 ) 



( 21 ) 



and 









1/2 



^-1 



v iL 

L r \. 



3=1 



( 22 ) 



The course of study selection thresholds do not change, so that 

the number of students of type i in a given course of study changes if they 

are to the right of the corresponding threshold changes. The change in the 

th 

number of students of type i choosing the j course of study if the choice 
is made k grades after application of the supplemental program is 
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(23) 
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and where u , = -oo and u , = oc; and, also 

P, k o,k 

where r-1, 

r <$ (v) - 

-1: I 1 ( -l/2)x * 



J" (v) is given by 



S-CO 



m 2 rr 



e 



dx = v 



(24) 



(25) 



and equations (18)- (21) can be used to calculate the achievement moments. 

Equation (23) states that the change in the number of students 
of type i in course of study j is related to the area under part of a unit 
normal distribution. The upper and lower limits of integration which 
define the area are given by (24); they are chosen so that the achieve- 
ment threshold values remain constant even though the moments of the 
distribution change. Students of type i with an expected achievement 

between the thresholds for the j 11 and (j-1) courses of study are 

th 

assumed to be in the (j-1) course of study; the proportion of students 
in that course of study before application of the supplemental program 
is known and is used to define the threshold achievement for that course 
of study. After the supplemental program has been applied, the moments 
of the distribution change. Using the same thresholds and the new 
distribution, one can compute the proportion of students in each course 
of study. The restriction that the total population be unchanged is met 
because 

Am^ = 0 (26) 

3=1 

by (19)- (25). . - 




. 155 • 



When the course of study selection does occur, it is assumed to 
occur before truancy, dropout, and net transfer computations are made, 

6 . Projected Impact on the Community: Present and Future 

6 . 1 The Equality of Educational Opportunity 

The equality of educational opportunity can be studied as a 
function of the difference in expected achievement for two student types 
at grade k, standardized by the standard error of the difference: 




77 



kCTkCT k , 



(27) 



computed before and after the application of a supplemental program. The 

* L j. • * — - 

quantity ^ represents the correlation between A^ and A^ and is set on the 
basis of empirical findings or judgments. The consequence of under- 

X J 1 

estimating to find equality of educational opportunity when it does 

not exist. The quantity dV^is distributed as a random variable with a 

El, 



t- distribution with 



nvjj-l degrees of freedom, and can be tested for 

J=1 



significance at a level by using the t- distribution. The test should be 

iL 

considered approximate unless is known. If w pairwise comparisons 
(26) are to be made, then, to ensure that the significance level is fixed 
at G* overall, each test should be made at the / w significance level. 

6. 2 The Number of Graduates 

The number of graduates is determined recursively from the 
balance equation (3) for the largest grade considered, after dropouts, 
and truants, have been computed in sections 4 and 5. 

6. 3 Dropout Rate 

The unconditional dropout rate is defined as the ratio of students 
dropping out in grade k to the total number starting grade zero. This 
measure is useful only if the transfers into and out of the school balance 
each other. A conditional measure of dropout is defined as the ratio of 
students in grade k who drop out to the number who started grade k. There 
is a simple relationship between the conditional and unconditional measure: 
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if D 1J = n 1J 
1 k k, dropout 



m 1 ^ , and c£* = nj? 
o k k. 



k, dropout 




if net transfers can be neglected. 

6 . 4 Expected Lifetime Earnings 

From the achievement predictions, dropout rates, and course 
of study distribution, it is possible to determine the educational levels 
of students joining a community, to predict what type of occupation and 
hence what expected lifetime earnings are to accrue to a student of a 
given type. This information is then related to the distribution of stu- 
dent types in the community being considered, and an expected average 
earning potential for a student in that community calculated as a 
weighted average of the earnings for a student typo weighted by the pro- 
portion of students of that type. 

7. Summary 

This summary has attempted only to describe the facets of the 
educational process which the difference equation model considers, and 
the final forms of the equations. Much of the motivating sociological 
theory has been omitted for space considerations, as have the often 
tricky problems of estimating the parameters and baseline statistics 
from the historical records. 
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APPENDIX B 
VARIABLE TABLE 



The variables listed in this appendix are those used in delineation 
of and illustrations from the mathematical submodels described in 
Chapters V - IX. 

The Variable List for Chapter V, The Instructional Process 
Submodel, is divided into sections that closely correspond to those in 
the chapter. 



Service Environment Variables 

A S is an index of the value of service which the Title I program 

provides 

a l» s 2> s 3 are coefficients relating A-Q & , Al g , and &D g , respectively, 



to A S. 

Quality of Service 

A O s is an index of the change in the quality of a school’s service 



environment 

P is a yes -no valued variable (written, mathematically as 1 or 

0) which is yes if a Title I service program provides a kind 
of service which is new to the target group; no, otherwise. 

F is yes valued if the service program is provided free of any 

monetary charge to the target group; no valued, otherwise. 

Of* ^2 are coe ^ici e nts relating P and F respectively to AQ g . 



Quantity of Service 
Intensity 

Ai s is an index of the change in the intensity of service which 

incorporates 5 factors. 

A#' s of professionals /student 
Asq. ft. of space /student 
. A$ spent on materials /student 

*l , *2 , *3 are coefficients relating the number of professionals, the 
square feet of space available for service activity, and 
the cost of service materials to Ai„. 



each of these represents 
a change caused by a Title I 
program that is incorporated 
into A I< 



L s* 
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*4* *5 are ac ^itional coefficients which relate, respectively, the 
conditions of low professional service quality and meager 
service sufficiency to ^1 . 



H. 



D 



h ; 



D' 

s 



Duration 

A D s an index of the change in time which students spend in 

service activity. 

represent the hours per day spent by target group in service 
activity before and after Title I, respectively, 
represent the number of days in a week which the target 
group spends in service activity before and after Title I, 
respectively. 

Ws. W s represent the number of weeks in a year in which the target 

group participates in service activities before and after Title I. 
d' is a coefficient relating the changes in service activity time 

to A D . 

i g 

Instructional Environment Variables 



Ac, Acj 



c , . 

l r 



are repsectively an index of the change in instruction, an 
index of the change in instruction in a specific, the jth, 
academic subject, 

c 2 j, c 3 . are coefficients relating /\,Q, A^I, and D, respectively, 

to A c j- 

j is an index for the academic subjects. 

Quality of Instruction 

A Q index of change in quality of instruction 

/^REC index of change in curriculum recency measured in change 
in years of textbook publication dates 

Ao t represents the change in teachers' salaries per teaching hour 
per student per year 

q 1} q 2 coefficients relating curriculum recency and teaching quality, 
respectively, to /\Q> 

ft threshold value linking A.REC and A Q, to A Q. 

L . j 

Quantity of Instruction 
Intensity 

A 1 an i n dex of the change in the quantity of instruction 






i 

«* 



li. 

«it 



tf. 



a 



h h 

W » 

.1 



t \i 

r H 

t jj 

a. 



r * 

ill 



i 



u 



u 



0 

0 






each of these represents a 
change caused by a Title I 
program that is incorporated 
into the Al index. 



/\ #'s of teachers /student 
/\ #'s of text’s /student 
A #’s of desks /student 

i- •••* 

f\ $ spent on aids /student 
*1»*2»*3»*4 coefficients relating the change in teachers per student, 

texts per student, desks per student, cost of instructional 
materials per student to /\ I. 

*5»*£»*7 are coefficients which relate the conditions of low teacher 
competence, scanty instructional material, and large 
classes to A I. 

Duration 

IS ^ an index of the change in the time to which students are exposed 

to instruction. 

H, H' represent the hours per day .in which the target group is exposed 
to instruction before and after Title 1, respectively. 

D,D' represent the number of days per week in which the target 
group is exposed to instruction before and after Title I. 

W, W represent the weeks per year in which the target group is 
exposed to instruction before and after Title I. 
d is a coefficient relating the changes in the time spent in 

exposure to instruction to /\ D. 

Student Impedance 



$ 

E i 

H 



S 

L 



L 

s 

D. 
i • 



a student's ’’impedance”, an index of his resistance to instruction, 
parental income level 
parental educational level 

measure of degree to which a student is physically or mentally 
handicapped. 

index of family solidarity 

the cultural lag- -the average amount by which the entire 

target lags behind national norms in academic achievement 

the student lag--average typical student's gap in basic skills 

for values of i from 1 to 5, an alternative representation for 

the first 5 factors of impedance, $, E , H, S, L . 

P c 
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z^ coefficients relating, respectively, parental income 
z z^ parental education, student handicaps, family solidarity, 
cohort lag and student lag to student impedance, 
a a normalization constant relating the D^'s and L g to Z. 

Project -Student Interactions 7 

A Z index of the change in student impedance 

E a measure of the effectiveness of a service program in 

s 

reducing student impedance 
G the target group's grade level 

Rp R2, R3 factors which relate Title I projects 

R4, Rg,R^ qualities to the factors of disadvantagenes s and thus to 
A Z; these relate, respectively, parental income, 
parental education, student handicaps, family solidarity, 
cohort lag, and student lag. 



A A 



Zp z^, z^lvSee under "Student Impedance" 



z 4> V z 6 



M the maximum amount of change that a project can cause in 

one year in student impedance. 

Zmax the upper bound of the index Z. 

Z m in the lower bound of the index Z. 

the change in academic achievement resulting from a Title I 
program. 

lag the number of grade levels the typical target group member 
lags behind the grade norm in academic achievement. 

F. the factor in the j\ A computation which accounts for the 

interaction between Z and /\C. 

F g the factor in the A, A computation which accounts for the 
interaction between Z, A Z , and A*S. 
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The symbolic variables listed here are those appearing in Chapter VI, 
The School Flow Submodel. These are those terms actually appearing in 
the explanation of the school flow model; the terms which are used in the 
introductory explanation of Markov chains are not included in this list. 

Term 

r the number of subjects in which a student is graded in any 

one academic term. 

i represents a particular pattern of academic performance 

expressed as a binary integer, 
t grade level, e. g. , t = 3 for the third grade. 

a student in state "s 11 has a certain combination of academic 
performance levels distributed across a certain set of 

* j* 

academic subjects. More specifically., a student in state s^ 
is in grade t and has the ith pattern of academic performance 
levels among r subjects. 

t- 1 t 

the probability that a student in state s^ will go to state s j . 
a matrix whose elements are the, set of P**(t) for 1 li Jr r and 
1 1" j Jr r; thus, the matrix contains the probability of passing 
from any state at grade t-1 to any other state at grade t. 
is a function equal to the sum,, to the base 10, of the digits 
in the binary integer i. 

is the average number of subjects passed per student in grade t. 
is the expected avhievement at grade t. 

is a vector whose elements contain the probabilities that a 
student will be in any given state at a given grade t. 
is an element in giving the probability for the s. th state 
is the number of t's. - 

standard deviation of the number of courses passed. 



P • -(t) 
ip 1 

p 

t 



Z(i) 

z 

A. 



P t < s i) 

T 



(Tt 
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This section lists the variables found in Chapter VII, The Dropout/ 
Truancy Submodels. 



k 

A 



n 



n 



k 

d 

k 

t 



B 



11 



B 



12 



d v 



AX 




D k 



grade level, e. g. k = 3 for the third grade 
achievement level at grade k 



number of dropouts in grade k 



number of truants in grade k 
coefficient relating .^n^to ^ ^ 

N t 

coefficient relating nj^ to /, ^ 



an expert guess as to the incremental decline in the number 
of dropouts that a one grade level rise in their achievement 
could cause. 

an expert guess as to the range which should be assigned 
to to be 50% sure that its n true n value is included in 
the guess. 

dropout rate conditional upon those starting grade k 
number of students who enter grade k 
number of net transfers and deaths during grade k 
dropout rate computed using as starting point the impact 
grade of Title I program. 
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This section contains those variables appearing in Chapter VJII, 
The Course of Study Submodel. 

k grade level e. g. , k = 3 for the third grade 

*^k J ac ^ e vement levels in the kth grade, before and after 

Title I, respectively. 

^k* ^k standard deviation of achievements in the kth grade, 
before and after Title I, respectively. 

P courses of study index 

N the total number of students to be distributed across all 

courses . 

n p> n p number of students in the pth course of study before and 
after Title I, respectively. 

m p maximum number of students in the pth course of study; 

such maximums occur because of limited school facilities, 
* *i^p~l course of study thresholds, the average achievement 
level which a student must obtain to be admitted to a 
particular course of study. 

Tp the number of students who were not able to be assigned 
to the pth course of study because of various constraints. 
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etc. 



The variables in this section appear in the Index of Equality of 
Educational Opportunity section of the Community Effects Submodel, 
Chapter IX. 



n 

i,k 

j 

A ij 



A* 



^2 ^2 
(J ij or Cf 1 






fikj 

e ik U) 



the number of students 
student type indices 

grade level--e. g. , j = 3 for the third grade 

the expected achievement at grade j for student type i, 

before Title I 

expected achievement at grade j for student type i, after 
Title I 

the variance of achievement distributions for the ith (or kth) 
student type in grade j 

the correlation between achievement scores between the 
ith and kth student types in the jth grade 
is a measure of the change in educational opportunity for 
student types i and k from the first to the jth grade 
indicates the significance of change in education opportunity 
as measured by e., (j) 
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CHAPTER I 
INTRODUCTION 



This volume is intended to depict some of the uses and the 
requirements of the Office of Education Cost- Effectiveness model, 
developed by Abt Associates. It should be pointed out that the model 
discussed in this report has not yet been programmed and put into 
operation. This model has been developed by Abt Associates to aid 
in the evaluation of alternative educational projects within school 
districts proposed under Title I of the Elementary and Secondary Ed- 
ucation Act of 1965. The specific intent of the model is to provide 
data on the probable effects of different projects that might be intro- 
duced into a school district. These data are not predictions of the 
absolute changes that the projects will cause, but are, instead, indi- 
cators of the likely effectiveness of the proposed projects relative to 
each other. The availability of such information to the skilled and 
experienced educational planner should enhance his ability to select 
and design projects that will increase the educational opportunity of 
the disadvantaged child. 

In broad terms the model functions in the following way: using 
information about the current school system, the historic performance 
of selected student subpopulations, the social, and academic character 
istics of the target population, and the Title I proposed changes in the 
school environment, the model computes likely short-range changes 
in students' educational achievement and attitude and then extrapolates 
these effects into longer- range changes in academic achievement, 
dropout and truancy rates, and certain community effects. 

The overall model is divided into four parts, each of which 
has a separate function; associated with each of these functions is from 
one to four submodels as indicated below: 
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FUNCTION i 


SUBMODEL 


INPUT 


. COST /INPUT 


IMMEDIATE TITLE I 


INSTRUCTIONAL 


EFFECTS 


PROCESS 


LONGER-RANGE 


SCHOOL FLOW 


EFFECTS 


DROPOUT & TRUANCY 
CALCULATION 




COURSE OF STUDY 
SELECTION 




COMMUNITY EFFECTS 


OUTPUT 


EFFECTIVENESS 

OUTPUTS 



Because a number of the central concepts of the model's 
present design are embodied in the Instructional Process (IP) and 
School Flow Submodel, their operation and theory will be discussed 
first. The Instructional Process Submodel calculates the changes in 
academic performance levels caused by a Title I project during the 
year or years in which it is applied; on the basis of these new levels, 
the School Flow Submodel extrapolates student achievement out to later 
years. The model calculates the extrapolated changes relative to its 
own "baseline". The baseline is generated by running the model with 
only historical achievement data. Thus, throughout this discussion, 
"change" refers to those increases or decreases which Title I pro- 
jects cause relative to baseline results. 

The computation done by the Instructional Process Submodel 
incorporates two main variables: the change in the overall effective- 
ness of the instructional environment and the change in the students' 
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responsiveness to the learning process. Among children with dis- 
advantaged backgrounds,' there is often some hostility toward the 
school environment and resistance to the learning process. This 
negative disposition is referred to as impedance toward learning 
in the model. 

The IP uses these two variables in distinguishing between 
the curriculum and service components of a Title I project. The 
overall curriculum change variable takes into account the values of 
descriptors of the changes resulting from a Title I project in in- 
structional quality and quantity. The calculation of change in im- 
pedance to learning is accomplished by analyzing a project's service 
components for their relevance to the individual factors of dis- 
advantage. The model attempts to determine whether a service 
provided by the Title I project will tend to make up for a background 
disadvantage of the student. If there is such a service, the student 
impedance will decline and achievement will improve. Should a pro- 
ject offer no service at all or service that is irrelevant to the dis- 
advantaged students' needs, then any improvement in achievement 
levels will have to come from the curriculum .component of the project. 

For example, a program which provides free lunches for 
children who have a history of low family income is deemed relevant 
since it tends to offset one of the disadvantages of these children. On 
the other hand, a program to provide eyeglasses for children who 
already see well on the average will not be relevant to any of their 
factors of disadvantage and will not decrease their impedance toward 
learning. 

Summarizing, the Instructional Process Submodel predicts a 
change in achievement based on the changes in the school's instructional 
and service environment, calculating this change up to the time; when 
the Title I program no longer is being operated for the particular target 
group in question. At this point, the longer-range effects portion of 
the model takes over. The first submodel in this part of the model is 
the School Flow Submodel, which traces the achievement patterns of 
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the students through the rest of their scholastic career up to the point 
where they either drop out or graduate from high school. The School 
Flow Submodel indicates the pattern of achievement for a group of stu- 
dents in any grade based on two factors: the achievement pattern for 
the group in the last grade that the IP calculated achievements for, and 
a set of probabilities describing the likelihood of a student moving from 
a particular pattern in that grade to a particular pattern in the next. 

For instance, one of the probabilities might be described by saying, 

"If a student in this particular community or type of community passed 
English and math in grade 4 but failed science and social studies, the 
chance that he will pass English, math, and social studies but fail 
science in grade 5 is . 03. " 

Thus, if there is a Title I program in the third grade, the In- 
structional Process Submodel will describe the achievement change 
for students in the third grade. Starting with the fourth grade, the School 
Flow Submodel will extrapolate, from year to year, the achievement 
changes for this group of students, keeping track of them all until they 
either graduate or drop out. 

It is likely that the educational analyst will be considerably 
interested in each of the submodels as a separate entity. Included 
in this volume in Chapter V is a discussion of the operation of these 
and of the other submodels as modules which can be separately 
investigated and tested and their coefficients adjusted. 

The achievement changes that the IP and School Flow Sub- 
models compute are key inputs to the remaining submodels in the 
long range effects portion of the model. These models combine these 
expected changes with historical information to produce extrapolations 
of dropout and truancy rates, course of study selection, earnings po- 
tential, and the index of educational opportunity. Some of the details 
of these extrapolations are discussed below. 

The Dropout/ Truancy Submodel calculates the long-range 
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effects of projected Title I programs on dropout and truancy rates. 

At the end of each simulated school year this submodel determines 
the number of dropouts and the average truancy rate for that year. 

The truancy rate is recorded so that it may be reported and the drop- 
outs are removed from the group of students whose school performance 
is to be extrapolated through the following years. The point at which 
students drop out is recorded and used both as an output and as data 
for the calculation of potential lifetime earnings. 

The Course of Study Submodel uses the projected achieve- 
ment changes to calculate the changes that will occur in the numbers 
of target group members who could be expected to be in each type of 
academic course in secondary school. That is, the submodel deter- 
mines how many of the students will go into the various courses of 
study, where available. 

Briefly, the procedure by which this submodel determines the 
changes in patterns of course of study selection is to calculate the 
changes in the mean values of achievement for students at the point 
in their school careers at which they select a course of study. Based 
on the upward shift in the distribution of achievement for the students, 
the model makes more students eligible for those courses of study for 
which a higher achievement mean is required. The course of study 
selected by the students as a result of the change in their achievement 
means will depend upon their backgrounds and the kinds of factors 
described in the Instructional Process Submodel. 

The outputs of the preceding submodels are presented to the 
user for analysis, but they are also needed as data for the Community 
Effects Submodel. There are two kinds of outputs to be derived from 
this submodel. The first kind of output. deals with the expected or po- 
tential lifetime earnings of the students, and the change in these figures 
as a result of the proposed Title I programs. A student's potential 
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lifetime earnings are calculated on the basis of three important variables 

1) whether he drops out of school or not, and if he does, when he does; 

2 ) the course of study he has selected in school, if the point of special- 
ization has been reached; and 3) for graduates, their achievement levels 
upon graduation. The second kind of Community Effects output is an 
indicator which describes Coleman's concept, "the equality of educa- 
tional opportunity. " That is, the association of student performance 
with student background is measured. To the degree that such an 
association is not present, to that degree there is said to be equality 

of educational opportunity. 

The remaining submodels, the Cost/Input and Effectiveness 
Outputs, are concerned with interfaces between the model and its users. 

The input portion of the model is a straightforward data input 
and error checking procedure which serves to construct the data-base 
for the model. It takes as input punched cards with data describing 
the particular school or school district, the student population, and 
the community as a whole. It determines whether cost subtotals add 
up to give the total described on the input cards; in addition, it checks 
for seemingly false implications among the data. For example, should 
a project be described as being relevant to the basic health needs of 
the target group and the Cost/Input Submodel cannot find any budget 
entries in the categories that are relevant to these needs, it will print 
out a note indicating the inconsistency and ask for confirmation or 
correction of this information. As well as reports of such errors or 
inconsistencies, the model provides reports on the proposed budget 
to facilitate cross-project cost comparisons and to allow' additional 
verification of these data. After requesting user clarification of am- 
biguous or incorrect information, the program will make up a data 
base tape for use by the submc J els described above durijng the actual 
simulation of the effects of the xitle I program. Detailed information 
on the data needs of the model may be found in Chapter III of this volume. 
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The last submodel to be discussed is the Effectiveness Outputs 
Submodel, which will provide output for study and analysis by the de- 
cision-makers involved in evaluating proposed projects. Because the 
model design has been aimed at developing an aid to decision-making, 
the output from the model must be in a form which its users can under- 
stand and work with. The output shows the effects of only a single 
particular program so that the general educational areas in which the 
program will have impact can be seen, and so that it can be determined 
whether the program will have beneficial or deleterious effects on the 
students in the school in question. More important, however, is the 
function of comparatively evaluating alternative proposed projects in a 
particular school or school district so that they may be readily analyzed 
by Title I decision-makers. A discussion of these outputs is presented 
in Chapter II of this volume. 

Although the Office of Education Cost-Effectiveness Model has 
not yet programmed and operated on a computer, its development has 
given rise to information about its probable use, requirements, and 
organization. This tentative information is found in this volume. For 
detail on the design of the model, Volume I should be consulted, 

A brief model profile is given on the next page as a summary 
for readers of this volume. 
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A Brief Profile of the OECE Model 



The Model Will: 


Function 


The Model Won't: 


Deal with groups of students 

Deal with students below 
national norms 

Indicate changes in student 
group achievement 


Students and student 
change 


Deal with individuals 

Evaluate programs to 
raise achievement of 
students above national 
norms 


Indicate changes in rate 
and year of dropouts 

Indicate increased or 
decreased numbers of 
high school graduates 

Indicate changes in course 
of study selection where 
applicable 


School change 


Simulate changes in the 
administrative policy 
in a school district 


Indicate changes in potential 
life-time earnings 

Indicate changes in equality 
of educational opportunity 


Community change 


Simulate change in the 
home as a result of 
Title I 


Compare the cost-effective- 
ness of proposed Title I 
and other educational im- 
provement projects within 
a school district 

Aid decision-makers 

Determine relative effective- 
ness of proposed Title I 
progra'ms 


Evaluation 


Compare proposed Title I 
projects across districts 
•* * « 

Make decisions 

Determine absolute 1 

effectiveness of proposed 
Title I programs 


Need commonly available 
data 


Data input 


Give results more precise 
than the input data 





CHAPTER II 

i 

USE OF THE OUTPUTS 



The outputs of the OECE simulation describe the changes 
in the students and their environment resulting from the 
introduction of a Title I program into the school or 
school district being simulated. The output formats presently 
designed are of several sorts. 

The Summary Output 

The overall summary shows the Title I program impacts.- on 
the school and community. Average Achievements, Values for 
the Index of Impedance to Instruction, Number of Dropouts, and 
Number of Graduates are listed under the heading School Impacts 
(Figure I). These are defined as: 

1. The Average- Achievement is the average of a student's 
grade levels in all subjects. It is given at the target grade 
and at the twelfth grade. (If the target group spans more than 
one grade, the average is given for the highest grade. ) 

2. The Index of Impedance to Instruction is a measure of the 
average incidence within a student type of certain character- 
istics thought to be detrimental to the student's receptivity to 
instruction. The index values range from one to ten; high 
values indicating that the student has many environmental 
characteristics of disadvantage, low values indicating fewer 
environmental disadvantages. 

3. The Number of Dropouts is the total number of students 
from the target group expected to leave school before grad- 
uation. 

4. The Number of Graduates is the total number of students 
from the target group expected to graduate. 

Equality of Educational Opportunity and Expected Average Lifetime 
Earnings are listed under the heading: 

COMMUNITY IMPACTS 

The Index of Educational Opportunity measures the degree to which 
students' educational achievement is dependent upon their socio- 
economic background. (A fuller discussion of this index is given 
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in Chapter IX of Volume I). The index in the summary is scaled 
so that if the condition of perfect equality is met, the index 
equals one; if perfect inequality occurs, the index equals zero. 

The Expected Average Lifetime Earnings show the projected 
earned income for an average student of a particular type. 

Achievements 

In addition to the information in the overall summary, more 
detailed data is provided about academic achievement (Figure 2). 
Achievements are given in grade levels; e. g. , a student reading 
at the beginning fourth grade level has an achievement level of 
4. 0 in reading regardless of the grade he is actually in. These 
levels are shown for the average student for each student type, for 
twelve grades, in up to six academic subjects. 

Dropouts and Attendanc e 

Outputs about the number of dropouts and the attendance rate 
are provided in a graphical format (Figure 3). These graphs 
give a twelve grade breadown for the entire school population; 
i. e, , the individual figures about dropouts and attendance for 
each grade are not disaggregated by student type. The attendance 
percentage is based upon the total student days, which equals the 
number of students multiplied by the number of school days. 
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SUMMARY OUTPUT 

I 

U.S.O.E. COST-EFFECTIVENESS MODEL 



• 9 • * 



COMMUN 1 1 r% FERNDALE ' 



PROJECT REMEDIAL 



• 




MASS 


i 


TYPE* 


READING 




TARGET 


G2-3 




ANNUAL 




t 


POPULATIONS 


SCHOOL 


B 


COST? 


85000. 


• 






SCHOOL 


IMPACTS 


- 




- 


STUDENT AVERAGE 


INDEX "OF 


NUMBER 


NUMBER 




TYPE 


ACHIEVEMENT 


IMPEDANCE 


OF 


OF 


* 




TGT-G 


G-12 


TO ...IN.S.TR , 


DROPOUTS 


GRADUATES 


BEFORE PROJECT* 


1 


• 1*2 


8*6 


8.6 


20 


28 . 


YEAR:; 1965 


2 


2'V0 


10V3 


6*4 


16 


31 




3 


r.4 


* 7 V.9 


8.0 


ef 


27 \ 




4 


3*2, 

<. * M • < 


12.0 


5*6 


9 


46 


AFTER PROJECT* 


1 


l«8 


"■ 10 * 2 


7.3 


16 


32 


YEAR: 1967 .. 


2 


2*6 


1 1 V0 


6*9, 


12 


35 




3 


' 2V/j 


1 1 V2 


5 *8 


17 


31 




4 


3*2 


12.0 


5.6 


C 


.47 



COMMUNITY IMPACTS 

EQUALITY OF EDUCATIONAL OPPORTUNITY 



BEFORE PROJECT# AFTER PROJECT# 

YEAR! J965 YEAR* 1967 



COMBINED ,40 
BY INCOME GROUPS #35 
BY ETHNGC. GROUPS ,18 



• 50 

• 40 

• 27 



EXPECTED AVERAGE LIFETIME EARNINGS 



STUDENT ‘ 
•TYPE.. 

1 

2 

3 

4 



BEFORE PROJECT# 
YEAR:' 1965 

S 200000 
S 300000 
i 325000 
S 450000 



AFTER PROJECT# 
YEAR* 1967 

S 215000 
$ 400000 
S 400000 
S 500000 



L 

. . STUDENT TYPES 

1 NON-WHITES UNDER 2000 INCOME 3 WHITES UNDER 2000 INCOME 
. 2 NON-WHITES OVER 2000 INCOME 4 .WHITES..OVER 2000 INCOME 
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ACHIEVEMENT OUTPUTS 

BY GRADE* SUBJECTS* AND POPULATION TYPES 



COMMUN I TY : FERNDALE 

MASS 



PROJECT REMEDIAL 
TYPE: READING 



TARGET G2-3 

POPULATION? SCHOOL'D 



ANNUAL 

COST: 



85000 



GRADE 1 



8 



10 11 



12 



STUDENT 






.. . GRAD.E 


LEVELS 


IN 1 


LANGUAGE 










BEFORE 


TYPE 

1 


• 6 


1*1 


1.6 


2.3 


3.0 


3*8 


4.5 


5.2 


7.0 


7.6 


8.2 


8*9 


PROJCT 


2 


• 8 


1.2 


1 .9 


2.9 


3*4 


4.5 


5.1 


6.0 


7.7 


8.4 


9.3 


10.0 


YEAR: 


3 


• 8 


1.3 


2.0 


3*1 


3.7 


4*9 


5*8 


6*6 


8.6 


9.2 


9.9 


10.9 


1 965 


4 1 


• 2 


2*1 


2.9 


4.0 


5.0 


6.2 


7.1 


8.0 


9.3 


10.2 


11.0 


12.0 


AFTER 


1 


*6 


l v *4 


2.9 


3*0 


3.9 


5*8 


6.7 


7.6 


8.5 


9.4 


10.3 


11.0 


PROJCT 


2 


• 8 


1.5 


3.0 


4.0 


5*0 


6*0 


6.9 


7.9 


8.9 


9.9 


10.9 


11. 


YEAR* 


3 


• 8 


1.7 


3.1 


4.0 


5.2 


6*2 


7.1 


8.0 


9*0 


10.0 


11.0 


12*0 


1 968 


A 1 


• 2 


2.1 


2.9 


4.0 


5*0 


6.2 


7*1 


8.0 


9* ‘3 


10.2 


11.0 


12.0 



6 



S.TUDENT 

TYPE 



GRADE LEVELS IN MATHEMATICS 



BEFORE 


1 

• t 


• 5 


1*1 


1.8 


2*3 


3.0 


3*6 


4*4 


5.0 


5.8 


6*3 


7*0 


7*7 


PROJCT 


2 


• 8 


1.3 


2.2 


2V9 


4*0 


4.6 


‘ 5V5' 


6 V 1 


7V0 


8*'0 


8*1 


8**8 


YEAR* 


3 


• 9 


1.5 


2.7 


3* ! 


4V 1 


SV0 


6V7 


6*6 


7*2 


8*0 


8V1 


8*8 


1 965 


4 


1 .0 


2.0 


2V9 


3*9 


5V0 


6.2 


7.1 


8.0 


9V1 

* *• # 


10*0 


11*1 


12.0 


AFTER 


1 


• 5 


1.3 


2*2 


3.0 


4.0 


4.6 


5.4 


6.0 


6*7 


7*2 


8*0 


8*8 


PROJCT 


2 


•8 


r*8 


2.9 


3*9 


5V0 


5*6 


6V5 


7* 1 


8*1 


8*4 


8*5 


9*‘0 


Y EAR * 

A At A At 


3 


• 9 


2.0 


3.0 


4*1 


5.0 


6.0 


7*0 


8.0 


8*8 


9V0 


9V: 


9Y4 


1 968 


4 


1.0 


2.1 


3*0 


* 4V2 


5*1 


6.0 


7*0 


8*0 


9*0 


10V0 


11.0 


12*0 


STUDENT 




. .GRADE. LEVELS IN SCIENCE 










TYPE 












• 




* 




•< 






BEFORE 


1 


.6 


l* 4 


2*9 


3.0 


3.9 


5.8 


6.7 


7*6 


8*5 


9*4 


10.3 


11*0 


PROJCT 


2 


*8 


tvs 


3V0 


4V0 


5V0 


6*0 


6V9 


7V9 


8*9 


9*9 


10*9 


n*6 


YEARS 


3 . 


• 8 


i v 7. 


3VI 


4V0 


5V2 


6V2 


7*1 


8V0 


9*0 


10*0 


11*0 


12*0 


196S 


4 


1.2 


2Y1 


2*9 


4Y0 


5Y0 


6**2 


7V1 


8*0 

* «•» 


9*3 


10.2 


1 1 V0 


12.0 


AFTER 


1 


• 6 


1*1 


1.6 


2*3 


3*0 


3*8 


4.5 


5.2 


7*0 


7*6 


8*2 


8*9 


PROJCT 


2 


• 6 


r.2 


IV9 


2V9 


3V4 


4V5 


ST1 


6*0 


7*7 


8’*’4 


9*3 


10V0 


YEARS 


3 


• 8 


1.3 


2V0 


3**1 


3V7 


4.9 


5.8 


6*6 


8V6 


9V2 


9*9 


10*‘9 


1 968 


4 


1V2 


2Vi 


2.9 


4V0 


5V0 


6.2 


7V1 


8*0 


9*3 


10V2 


1 1 V0 


12*0 


S T'UDENT TYPESS 


1 -NONWHITES# 


LESS 


THAN 


2000 


3-WHITES# 


LESS 


THAN 


2000 




* 




2-NONWHITES# 


MORE 


THAN 


2000 


•4-WHITES# 


MORE 


THAN 


2000 





'FIGURE, 2. 



12 



o 

ERIC 



* , 















O 

ERIC 



ATTENDANCE AND DROPOUT OUTPUTS ■ 
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There are three basic uses of the outputs from the simulation. 

The first and most obvious usej is that of program evaluation, where 
the computer output is used as an aid to decision-making. The second 
use is that of checking the design and variable settings of the model. 

The third use is for the redesign and new design of computer outputs 
which will better accomplish the first two tasks. 

The evaluation function is carried out through the comparison 
of the values of the variables shown on the output forms for different 
potential Title I programs. These data are in a form readily 
usable by educational adminstrator s and experts, and the comparison 
of several potential Title I programs should be fairly straightforward. 

The variables described above are present, and the different 
programs will be likely to have differential effects with respect to 
these variables, so that one program might raise achievement 
significantly in a particular subject area, while another program 
might raise achievement slightly in all of the available subjects. 

There was some thought given, at the beginning of the design 
project, to the development of a system whereby the various changes 
in the different variables might be assigned values so that each 
program might then be characterized by a single value number. 

This value number would be the total of the values for the various 
changes. It was decided, after consideration of the problem, that 
this form of benefit calculation would be inappropriate for two 
reasons. First, the determination of values for the changes in 
variables is not a simple problem. It is highly probable that the 
value of a change in one variable is not independent of changes in 
other variables. Secondly, there is no opportunity for' educational 
experts to use their experience and expertise when dealing with a 
single number; the computer, using a simple mathematical model 
(simple when compared with all the complexities of reality), cannot . 

begin to bring to bear the kind of intuition and experience which the 
analyst can draw upon when making a value judgment. • $;V!“ 

It was therefore decided to give the decision-maker direct access 
to the computed expectations of the model (in terms of results of 
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Title I projects) and the associated cost ^figures, and thus enable him 



to make his own determination of the relative values of the overall 
I projects based on his experience and judgment. 

The second main use of the simulation outputs is that of 
I checking the parameter settings and logical design of the model. 

When the results of the simulation seem to disagree consistently 
with the user's judgment as to the likely effects of the potential 
program being simulated, an investigation should be made.' When 
results from the field on an actual program are available, they 
may be used for comparison purposes, with the realization that the 
communities in which the two similar programs were attempted or 
simulated were, of course, different. In addition, the literature 
may be searched for references to similar projects and their 
effects, the judgments of other experts- may be consulted, and 
research projects in that area may be found relevant. 

Based on this type of investigation, the user may decide to 
make no changes in the model, or he may wish to change the setting 
of one or more parameters, or he may wish to change the model's 
logic. The change may be checked by modular operation of the 
submodel in question (see Chapter Y). If it is found to give 
better results than the previous setting or design, it should be 
retained in the model. 

When dealing with an area as unexplored as that of large-scale 
simulation of the educational process, we must be careful to allow 
room in the design for such changes as those described above. 

This has been accomplished in the present model by 1) structuring 
the outputs so that the user sees the results of a simulation run in 
a familiar form and thus can detect model results which disagree 
with his intuition, 2) by designing the model in modular way so that 
whole parts may be removed and replaced, 3) by using parameters 
in such a way that their values may be changed easily and that their 
effects are clear, and 4) by planning to program the model in a 
widely known and available and easily used computer language, 
Fortran IV. 
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Routine use of the simulation may point up deficiences in the 
output format. Since the simulation itself is only as good as the 
information it conveys, suggested revisions of the present output 
presentation will be as important as suggested revisions of 

* I 

the model itself. A simulation output is supposed to give the 
user the information he needs for decision-making in a form which 
best suits his needs and which can best be understood by him. At 
times, the designers of simulations assume that if all the important 
variables are listed in the output, the format of the output does not 
matter. An example of this sort of thinking is the usual standard 
regression package output; all the variables are present, but a • 
user who is anything other than expert cannot make very good use 
of the data. 

A test of the sufficiency of the present outputs, which were 
developed in concert with members of the Title I group at the 
Office of Education, will be: are the users of the outputs, in 
their study of the data presented therein, constructing new graphs, 
matrices, tables, etc. to aid their evaluation? If this is the case, 
and the auxilliary aids being used are of a standard form, then 
they should be programmed and added to the library of available 
output s . 

These then are the three basic uses of the simulation outputs 
program evaluation, parameter setting and logic redesign, and 
output format redesign. 
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CHAPTER III 

j 

INPUT DATA REQUIREMENTS OF THE MODEL 

The overall model needs three kinds of information derived from the 
proposal request questionnaires: information describing the characteristics 
of the student, the characteristics of the school, and the specifics of the 
project. These three sets of data are retained for use by the computer. 

The information needed will be provided by the requesting party on a 
questionnaire form accompanying the standard Title I request form. 

This questionnaire consists of three major parts, each concerned 
with one of the types of data needed to evaluate projects proposed under 
the provisions of Title I of the Elementary and Secondary Education Act 
of 1965. The first part is a set of questions about the general character- 
istics of the school district and its student population. In general only 
one copy of this section need be completed regardless of the number of 
proposed projects. The second and third parts contain questions about 
the details of the proposed projects and the characteristics of the 
student body subpopulations (or target groups) toward whom the projects 
are to be directed. For each proposed project a Part II form must be 
completed and for each unique subpopulation a Part III form must 
be completed. Figure I illustrates this procedure. 
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FIGURE I 



Proposal Request Questionnaire (Example) 



Part One 



Part Two 



Part Three 



General Oharacteristios Details of the proposal Student body characteristics 
oi the School and the Projects 

Student Population 



Part Three 



Part One 



One copy for each 
school district 




Part Two 



Project B 
Free lunch 
1st grade 



I 



i 

Part Two 



3rd Grade f 





Part Three 



5th Grade 



Part Three 



1st Grade 



etc. 
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Since the thrust of the ESEA is to improve the education of the 
disadvantaged child, evaluation of proposed projects must necessarily 
take into account known factors of disadvantage of the projects' target 
groups. Thus, a large proportion of the data to be entered in this 
questionnaire is concerned with the socio-economic backgrounds of 
the students and how these backgrounds are related to academic per- 
formance. Throughout the questionnaire, information is requested 
about four socio-economic types of students. Student types are de- 
fined so that we may separate those groups which we expect have 
significantly different backgrounds on the average, or which we expect 
may show differential change as a result of Title I programs. The 
types presently used in the model are: 

Type I, non- white students from families with an annual 
income less than $2, 000. 

Type 2, non-white students from families with an annual 
income greater than $2, 000. 

Type 3 , white students from families with an annual income 
less than $2, 000. 

Type 4, white students from families with an annual income 
more than $2, 000 o 

If precise data are not available for these student types, then whenever 
possible, reasonable approximations should be made. Different group 
definitions may be used where applicable, depending on the community 
in question. 

A project will have somewhat different effects on different students 
depending not only upon their socio-economic background, but also upon 
their school experience and age. To evaluate a project, these latter 
factors must be taken into account; this is done by describing the group 
at which the project is aimed in terms of the typical or average "target 
group" member. A "target group" is defined as a student body sub- 
population which has a unique combination of school environment and 
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grade. Thus, a school subpopu,lation in which all the members attended 
the same school but were' in two different grades, e. g. , third and fourth 
graders, constitute two "target groups". A Part III form must be 
completed for each of these groups. 

If a project is aimed at two or more different target groups, it 
may be necessary for the purposes of this questionnaire only to treat 
that project as two or more different projects. This is necessary when 
the data concerned with the effects of the project on the target group's 
environment is not the same for the different target groups. If these 
data are differ ent,__a Part II should be completed for each different ' 
data set, with the appropriate pro rata adjustments made on costs, etc. 

Each of the three parts of the questionnaire will now be discussed in 
detail- -Figure 2 illustrates the organizations of the questionnaire. 

General Student Body and School Characteristics (Questionnaire- -Part One) 

This part of the questionnaire asks for seven kinds of numerical 
data for each of the school grades and for each of the four types of stu- 
dents. The first three questions deal with the present number of stu- 
dents, the number of truants, and the number of dropouts. We assume 
that these data are .easily obtainable in total. We hope that the school 
administrator can estimate the numbers for each type of student. Many 
of these numbers can be left out (e.g. , dropouts, grade 2, all student 
types) since they are negligible. 

The last four questions pertain to course of situdy selections. In 
some school systems these questions are not relevant and can be omitted. 

We allow the requestor to specify up to four kinds of secondary courses 
of study, e. g. , college preparatory, general, commercial, and voca- 
tional. Some school systems may have no courses of study as such, some have 
fewer than four, some may have more or give them different names. Our 
model assumes that the student body is separated into at most four courses 
and that within each course numbers can be given for each of the four 
student types. 
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FIGURE 2 



ORGANIZATION OF THE QUESTIONNAIRE 



PART ONE: General Student Body and School Characteristics 

(Seven Questions) 

1. Question # 1 asks for the number of students by year and student type 
Question # 2 asks for the number of truants by year and student type. 
Question # 3 asks for the number of dropouts by year and student type. 

2. Questions # 4-7 pertain to the course of study (numbers of students 
and the achievement levels in each of four courses of study. ) 

3. All of the questions ask for information categorized by grade and by 
student type. 

Model Use : Dropout, truancy, course of study routines. 

PART TWO : The Title I Project Characteristics (Four Kinds of 

Questions) 

1. The category best describing the project (Tree lunch, TV lecture, etc.) 

2. Project impact on school instruction. (Changes in numbers of teachers, 
costs, etc., by academic subject area. ) 

3. Project impact on school services. (Changes in numbers of professionals , 
costs , etc. ) 

4. Project cost by category (administration, instruction, attendance, 
services, etc.) 

Model Use: Instructional Process routine. 

PART THREE; The Target Group (Three Kinds of Questions) 

» , 

1. Student background (parent education, family income, etc.) 

2 . Student achievement (by subject area) 

3, Student classroom' (by subject area) 

Model Use: Instructional Process and School Flow Routines 



Questions four through six in Part I require the school administrator 

t . 

to give the grade in which the student is assigned to a course of study, 
the numbers of students in each course, and the maximum number of 
students that can be accomodated in each course. This latter question 
implies that the physical layout of facilities and numbers of available 
staff to teach each course may be limited. 

Question number seven requires estimating the average achieve- 
ment for each of the four types of students. We expect that achievement 
will be expressed in grade years and months (with a ten-month school 
year), so if the average achievement level at the end of the seventh grade 
were 6.8, this would be interpreted as sixth grade, 8th month. These 

achievements should be estimated in relation to national norms where 
possible. 

The important fact to remember is that a proposed project which 
deals with several different grades or students from different schools 
. must be broken up into sub-projects -- one for each set of students. 

This distinction is necessary because of the possible differential effects 
of the program on the different groups. 

The Title I Project Characteristics (Questionnaire --Part Two) 

The second part of the questionnaire asks for qualitative and 
quantitative information in order to describe the Title I project, 
namely, a qualitative selection, from a list of five, ’of the category 
which best describes the proposed project; and the quantitative impacts 
that the project will make on instruction and school service. Each of 
these sets of information will be explained in more detail. 

The administrator is asked to choose, from the following list, 
the category which best describes the proposed sub-project: 

1. Does the project supplement the students’ need for basic 
necessities such as food and clothing? E.g. , a free lunch 
program. 

2. Does the project enhance the students’ academic program by 
adding or strengthening subjects? E.g., a TV lecture series. 

3. Does the project broaden the students’ cultural exposure? 

E.g., field trips, inter-school attendance, or the like. • 
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4. Does the project supply medical, dental, psychological, or 
therapeut ic services? 

5. Does the project provide remedial academic instruction? 

Any or all of these five categories may be offered as part of the 
proposed project. The purpose of the questions is to determine the 
degree of match between the project offerings and the needs of the 
students. 

The administrator is asked to provide quantitative data describing 
the impact that the proposed Title I sub-project has on the school in 
terms of changes in either numbers of teachers, desks, or updated 
curriculum materials, and the like. We have divided the project impact 
questions into an instructional section and a school service section. 
Within the instructional part we have further subdivided the questions 
into academic subject areas. We are interested in the changes in 
numbers of teachers or desks with respect to a particular elementary 
school subject such as mathematics, reading, writing, or spelling 
while for the secondary school, we are interested in the c hanges in 
numbers of teachers or desks for subjects such as mathematics, 

English, social studies and science. 

In section four of Part II of the questionnaire, the administrator 
is asked to provide cost data for each of his sub-projects. The 
categories for these costs are the same as those asked for in Form 
OE 4305 items 8 A and 8B. 

In summary, Part II of the questionnaire, dealing with the Title I 
project, is concerned with determining the characteristics of the 
project which are immediately relevant to the student's disadvantage. 

The Target Group (Questionnaire*- -Part Three) 

In Part III, questions are asked pertaining, first, to the students' 
background, next, their achievement, and finally, their classroom. 

Three questions are addressed to the students' background, as to 
whether the average parents have less than a secondary education, 
whether the family life is disrupted, i.e,, one or both parents absent 
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in the home and whether the students are physical ly or mentally 
handicapped. These questions set the stage for determining some 
of the characteristics of the students’ disadvantage and are asked 
separately for each type of student. 

We characterize the remainder of the indication of students' 
disadvantage in the second section of questions, by their academic 
performance in certain subject areas. The questions ask the administrator 
to list his estimate of the numbers of students in each of the four 
target groups who perform satisfactorily in some or all of the subjects; 

■ we ask these questions for each combination of subjects. For example, 
we are interested in how many elementary students did well only in 
reading and writing, implying that they did poorly in mathematics 
and spelling. Bypoorly, we mean well below the averages established 
for this local school group, in the eyes of the administrator. 

The third set of questions pertains to the classroom and 
specifically to the same four subject areas treated previously. We 
need to know: 

1. Whether or not the teachers' mastery of the subject is as 
complete as desired, 

• 

2, Whether or not the classrooms are large enough and materials 
in as great a supply as desired, 

9 

3, Whether or not there are fewer teachers than needed for 
desirable work loads 

4. Whether or not there are more than 30 students to a classroom. 

And in terms of the school services provided: 

$ 4 

5. Whether or not the quality of the professionals providing 
service is as high as desired and 

6, Whether or not non-classroom areas are large enough and 
service materials in as great a supply as desired. 

All of- these questions determine the degree of match between the 
proposed program, the students' needs, and the school classroom needs. 

Summary 

The three parts of the questionnaire have been discussed in terms 
of information needs of the model and the requirement that the school 
administrator provide characteristics of his school district, character- 
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the specific parts of the projects, as well as delineation of 
academfe variables* 

A preliminary version of the questionnaire follows. 
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proposed title i project 

QUESTIONNAIRE 



School Name 



School District 



City- 



County 



State 
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QUESTIONNAIRE 
PART ONE 



The General Student Body and School Characteristics 
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Answer each question 
(Yes or Nor* 



Target 6fot*p 

€»rade Level ^ _ _ 



1* the teachers' mastery 
oi the subject as complete 
as desired? 



Are classrooms as roomy 
and materials in as great 
supply as desired? 



Are there fewer teachers 
than needed for desirable 
work loads? 



Are there more than 30 
students per class? 
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A, 3 Number of Dropouts Per Grade* 

(If the number of dropouts for an 

graded population, the column for that grade mlfb^l^t blink) 
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If these statistics are not available by student type, please enter 
the total by grade, otherwise the totals should be left blank. 
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C*fc-tUr’f*sl Steffi fit Ctf # fitk lc ! 5 : iill€i 

A, 4 C*r4s tit Immediately After Which €&it?§e of 
.Stiffly i* Ifsmstly f !io§en 
(If school system does not have eoorses of 
study, enter a ?er© and ship remafrider of 
Part A) 



A# 5 Current Number of Students in F*mh Course of Study 



Courses of Study 
(Titles) 



Courses of Study 
(Titles) 



1 


Student Type 
2 3 


4 


* 




* 





Maximum Number of Students that 
Can be Accomodated by the Personnel 
Facilities Available for Each Course 



A. 7 Average Achievement in the Grade ‘ 

Preceding Course of Study Selection 

Student Type 
12 3 4 

Of the Top 25% of Students 
Of the Top 50% of Students 

Of the Top 75% of Students 

(Average achievement is expressed in years and month-s; 
e. g. , 6. 8 means 6th grade, 8th month) 
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PART TWO 



The Title I Project Characteristic* 
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B. J Project Name 



2 trom l e, f s ? n identification number 
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The project supplements the students 1 basic 
necessities such as food and clothing. 



enhances the students 1 academic 
program by adding or strengthening subjects. 

The project broadens the students 1 cultural 
exposure through field trips, inter-school 
attendance, etc. 

The project supplies medical, dental, 
psychological, or therapeutic services. 

The project provides remedial academic 
instruction. 
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*‘* 4 ,?>4 »•- ^mperiM with U»e changes^ that this projctt 

ret} l ‘ IV{ 7 ,n - 1 a g InstruefToMl environment, 
if file pro© of will eatifee measurable ©longer in the environ® 
nmmi factors in mlmm 1, please eniei the sissc ©f the 
tmrmve or d crease under the most appropriate subject 
m column % m (He© r eases should be enclosed in parentheses*} 

1 

Knvi ronmental El emenlar y 

Factors 1 Target 

Grade 

Secondary 
Target 
Grade 



Number of Teachers 

Number of Texts 

Number of Desks 

Available Classroom 
area 

Numb er of Years 
texts are updated 

Student Time in 
instruction 

Hours/Day 

Days/Week 

Weeks/Year 

Teachers* Salaries 

Cost of Curriculum 
supplies and 
equipment 



$ 



-- * - l" S£^K.;» I 


2 

a :*>«-- 1 «jg -■•»- -y ■ 






Math 

wta— me 


Reading 


W riling 


j Spelling 


Math 


English 


Social 


! Science 






Studies 


i 

K 

a 

i 


sq. ft. 


sq. ft. 


• 

sq. ft. 


sq, ft 


yrs. 


yrs 


. yrs. 


yrs. 


hrs. 


hrs 


. hrs. 


hrs. 


days 


days 


days 


days 


weeks 


weeks 


weeks 


weeks 




$ 


$ 


$ 




$ 


$ 


$ 
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H. b ^eettm B. S is eene unetl with the chances that this m-t.S. t * „,it 
jtave on »ts {gi^Ltsp^tgj^ service environment. If the nroieet 
will cause measurable changes In the service "environment Cut mi 
in column l below, please enter the amount of the mere t*-** 11#* 

r ,uma 2 - (Deerea8u8 “ be 



— !^ vice Fa etors (increases or Decrease* 


Number of Service 
Professionals 

Area Allotted for Service 
Activities 

Hours of Service Activity/Day 
Days of Service Activity/Week 
Weeks of Service Activity /Year 

Cost of Service Supplies 

and Equipment 
♦ 


aq, ft, 

hr a, 
days 
wedca 

$ 



B ’ 6 attr^uted ti°ntnvM at amou " t of tho Projects cost which can be 

* ,h ° **■« *«-» 



Administration — — ============ 

Instruction 
Attendance Services 
Health Services 
Pupil Transportation Services 
i Operation of Plant 
Maintenance of Plant 
Fixed Charges 
Food Services 
Student Body Activities 
Community Services 
Remodeling (Less than $2000) 

Equipment 

Professional Services for Site 
Sites and Site Additions 
Improvements to Sites 
Professional Services for Building 
Remodeling ($2000 or more) 

Equpment (obtained as part of construction) 

Total 
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QUESTIONNAIRE 
PART THREE 
B 



The Target Group's Social and Academic Characteristics 
and School Environment 



One copy ol this section must he completed for each target group 



-t— 7* -raw ? 



Target Group Characteristics 



A copy of this sectidn should be completed for each target group. 



C. 1 Target Group Grade 



C. 2 Assign this target group an arbitrarily chosen identification 
number from 1000 to 1999. Each target group should be 
assigned a different number. 



C. 3 Answer yes or no to each question below about the typical 
target group student's background. 



Student Type 



Do the student's 
parents have less 
than a high school 
education? 


1 

i 


2 


3 


4 


Is the average family 
disrupted? (E. g. father 
absent, mother working 
fulltime, etc.) 


1 

2 

i 

1 

i 

i 

I 






Is the average student 
physically or mentally 
handicapped? 




i 

1 

1 

i 

i 
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Complete section C. 4 if target group is at an elementary grade, 
section C. 5 if target group is at a secondary school grade. 



C. 4 Target Group Characteristics 

' * | 

Indicate the number of target group students of each type who 

have the indicated patterns of academic performance levels. 
(If neccessary, use estimates. ) 



Pattern for elementary 
school target grades 



1 . 

2 . 

3. 

4. 

5. 

6 . 



. 7. 



8 . 

9. 

10 . 

11 . 



12 . 



13. 

14. 

15. 

16 . 



Did well in all subjects 

Did well only in math 

Did well only in reading 

Did well only in writing 

Did well only in spelling 

Did well only in math and reading 

Did well only in math and writing 

Did well only in math and spelling 

Did well only in reading and writing 

Did well only in reading and spelling 

Did well only in writing and spelling 

Did poorly in all subjects 

Did poorly only in math 

Did poorly only in reading 

Did poorly only in writing 

Did poorly only in spelling 





Student Type 


1 


2 


3 


4 


Total 















Target Group Characteristics 



C. 5 



Patterns for secondary 


Student Types 


school target grades 


1 


2 


3 


4 


Total 



1. Did well in all subjects 

2. Did well in only math 

3. Did well in only English 

4. Did well in only social studies 

5. Did well in only science 

6. Did well in only math and English 

7. Did well in only math and social studies 

8. Did well in only math and science 

9. Did well in only English and social studies 

10. Did well in only English and science 

11. Did well in only social studies and science 

12. Did poorly in all subjects 

13. Did poorly in only math 

14. Did poorly in only English 

15. Did poorly in only social studies 

16. Did poorly in only science 



C. 7 The Target Group's School Environment 



Service Environment 



Answer the next two questions Yes or No. 

Is the quality of the service professionals 
as high as desired? 



Are non-classroom conditions as roomy 
and service materials in as great supply 
as desired? 



CHAPTER IV 



USER INSTRUCTIONS FOR OPERATING THE MODEL 

This' chapter describes the steps that the operator of the 
simulation, referred to hereafter as the Program Administrator 
will take to operate the Cost Effectiveness model. He performs 
two important tasks: First, insuring that the information needed 

by the model to evaluate alternative proposals is complete and 
accurate; and, second, submitting runs to the computer and dis- 
tributing computer output. Figure 1 illustrates these activities. 

♦ 

Data Collection and Preparation 

Preparation for a model run will start when a project request 
is received from the field. The project request will be accompanied 
by information describing the project, the selected students, and 
their school environment. These data will be submitted on standard 
typed forms. The Program Administrator's job will be to ensure 
that the information is complete and accurate. A Preprocessor 
Computer Program will be provided to help accomplish this part 
of his task. The field data will be placed on punched cards and 
checked by the Preprocessor Program for completeness and con- 
sistency. Errors discovered by the computer will be printed and 
returned to the Administrator for correction. In some cases, such 
as those of errors of omission or keypunch errors in spelling or 
the like, the Administrator can make the corrections himself. 

For more serious errors, such as a case in which total cost 
might be less than the sum of its parts, he may have to seek 
counsel from the proposing district. 

Several kinds of projects may be suggested in a single request. 
These can be either instructional projects or school service projects 
for one or more school districts. The Program Administrator will 
be responsible for sorting out the relevant descriptions of students 



I 



and school systems. Suppose, for example, that the request 
calls for a free lunch program for the first three grades at 
P. S. 12 and 13, and remedial reading for grades four to six 
at P. S. 18 and 30 in a particular school district. The Adminis- 
trator must be sure that separate information has been provided 
to describe the students in all six grades, as well as data about 
each of the four schools. This variety of information can be 
placed in the project data base, but the computer program will 
have to be instructed as to which child goes with which school 
(or classroom) for which project. 

The Administrator is responsible for organizing the request 
into its separate parts, and instructing the computer program as 
to how he wishes the project analyzed. He is also responsbile 
for coordinating his project organization with the evaluation 
group in order that the computer model output will be meaningful. 

Data Base Makeup 

Once having obtained all of the field information necessary 
to analyze the project request, the Administrator will have to 
choose a set of model coefficients which are consistent with the 
type of student and the type of school district. The "closeness 
oi fit of his choices will depend upon his knowledge of how the 
model works. The coefficients are used in the cost-effectiveness 
model to project student achievement forward in time and different 
sets of coefficients will be provided for different school environ^ 
ments such as urban versus rural districts. 

The Administrator will prepare his project organization on 
control cards. A separate control card will be provided for each 
separate part of the project request. 

The three kinds of information - field data, coefficient 
selections and control cards - will be given to the computer. 

The computer program will search its master coefficient file 



for choices and create a community data base file. This file 
is called the project file, and all subsequent requests for cost- 

effectiveness runs for this community can be made directly 
from this file. 

Model Runs 

Running the model is the Program Administrator's easiest 
task, AH that will be required will be the submission of his data 
base file with a request for machine time, since the file will 
contain all the control cards needed for model operation. The 
computer will make two passes for each separate part of the 
project request. The first pass, called the baseline run, will 
predict the consequences of the selected education system's 
operation without the presence of the Title I project. The 
baseline run is a prediction of the numbers and qualities of 
graduates, assuming that the school system remains the same 
as it is today. The second pass will be made with the Title I 
project included. The difference in results between the two runs 
indicates the specific impact made on the project. Data from 
each run will be collected and summarized in a separate schedule 
by the computer for the project as a whole. 

We have described the activity schedule of the Program 
Administrator, taking him through the three major tasks of model 
operation: data collection and preparation; data base makeup; 

and, finally, Model runs. These will be his principal operating 
activities. It is possible, however, for him to provide certain 
other services to the evaluation group. 

The model is designed to serve both as a production evaluation 
device, and as a research tool. This latter use requires, among 
other things, familiarity with the computer program, the data file, 
and the model coefficients. Separate parts of the model can be 
operated by a user upon the request of the evaluation group. 

In addition, the settings of the coefficients will be modifiable 
when, and if, research indicates that this should be done. The 
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Program Administrator will have to be trained by the model 
designers, Abt Associates Inc. , as the computer program is 
developed. In addition, the Program Administrator Manual will 
contain detailed instructions for both operating the model and 
updating the coefficients. 
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PROGRAM ADMINISTRATION ACTIVITIES 



Data 

Collec- 

tion 

and 

Prepara 

tion 



Source 



Field 



Data processing 
facility 

Machine 
proces sing 

Administrator 
error analysis 



Operation 

Fill-out question- 
naire 

Convert data to 
punched cards 

Run pre-processo 
computer program 

Determine correc 
tions needed 



Pictorial 



questionnaires 



s 

| j keypunch cards 



\y 



bp| editing 



4/ 

<i 



report of errors 



Data 

Base 

Makeup 



Administrator 
error correc- 
tion 

Run control 
preparation 

Machine 

processing 

Data 

processing 

facility 



Opera- 

tion 






Machine pro- 
cessing 






Administrator 



Prepare field data 
cards 



Prepare control 
cards 

Run Data Base 
makeup program 

Save data base 
tape 






* * u field data 




contrc 

cards 



Run cost-effec- 
tiveness computer 
program 

Save output tape 



N/-S Master 

Data Base Input \ * L c p e ^" 
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Print the output 
tape • 




Control 
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> — Output tape 
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Cost 
Effec 
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Compu 
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Figure 1 
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CHAPTER V 



modula r operation of the submodels 



Although the OECE model is conceived as a whole entity for multi- 
faceted analysis of proposed Title I programs in terms of scholastic, 
community, and economic effects, it will be possible to operate its 
various component submodels individually or in any logical combination. 
For actual program evaluation, as a part of the Title I fund -allocation 
decision process, the entire model will be used, but there are several 
situations in which operation of less than the complete model can be 
valuable. This being the case, we have designed the model so that it 
will be fairly simple to run its various submodels in isolation. It is 
expected that modular operation will be available on a time-shared 
computer system with a remote console at the Office of Education. 

For example, if one wished to see how projected lifetime earnings 
change with different 12th grade achievement patterns, this evaluation 
could be accomplished by entering as input each of the achievement patterns 
to be considered, operating upon them with only the Community Effects 
submodel, and comparing the computed projections. 



The following paragraphs discuss the possible purposes which may 
be served by such modular operation. These purposes fall roughly into 
two categories.- 1) -tuning" the submodels and 2) familiarizing users and 
potential users with the performance characteristics of the submodels. 
"Tuning 11 the Submodels 

Prior to any actual planning use of the model, it will be necessary 
to check the predictions made by each of the submodels and, if required, 
adjust individual parameters to give results which seem reasonable in 
response to a broad range of input data characteristics’ Finding the 
correct values to assign to these parameters will be a process of en- 
lightened trial and error. The model, which is no more than the sum 
of its submodels, has been designed primarily on the basis of qualitative 
data quantitative theory is sparse and, although it was used whenever 
available, contributed relatively little to the final model design. Conse- 
quently, although the model is based on the interaction of many diverse 



factors selected after careful study of the education problem, the exact 
magnitudes of many of these interactions are imprecisely known. That 
is, the model, after it has been programmed for use on a digital com- 
puter, consists of a set of interconnected submodels, each of which 
consists in turn of a number of mathematical equations interrelating the 
input data according to the same patterns which seem, on the basis of 
our research, to operate in the real world. But it is not always pos§!bt© 
to deduce from observations of tbe real world whether, of two dissimilar 
factors which clearly both affect the -same variable quantity of interest, 
the first factor has twice the effect of the second factor or has only one 
and a half times the effect, etc. For example, in the Instructional 
Process submodel we know only that, among other factors, change i«t 
teacher experience and change in recency of curriculum materials both 
are associated with changes in student achievement level 

The user has access to the parameters in the submodel equations 
which relate the two input variables to change in achievement and can 
set these parameters to any values he desires. In order to arrive at the 
correct relative weightings for change in teacher experience and change 
in recency of curriculum materials with respect to change In achievement 
level, the user will first set the controlling parameters according t© Ms 
best "informed guess, "--say that change in teacher experience it E 0 1 
times more powerful than change in recency of curriculum materials In 
terms of the associated change in achievement level, l,e , , a 10% Increase 
in teacher experience would have 2, 5 times flie influence of a 10% Increase 
in recency of curriculum materials. The user will then operate the 
Instructional Process submodel using a set of input data with which he It 
familiar- -it may be real-world data or he may have mad© It up just for 
the purpose of "tuning" the submodel. But in either case he should have 
a feeling for the results which reasonably would be expected according 
to the changes in teacher experience and curriculum materials recency 
specified in these data. If, comparing the submodels calculations with 
his intuitive expectations, the user finds that the submodel outputs 
contradict or do not quite match the results he has forseen* he can readjust 
the parameter settings and run the submodel again until he is satisfied 
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could thus familiarize himself firsM-by reading the appropriate chapters 
in tills report **\vltli the theoretical aspects of each submodel: the input 
data it requires, the outputs it generates, etc. He could then explore 
each of the submodels by introducing various sets of data to the submodels 



#ii*I observing the changes In the output in response to changes which he 
made In the Input* In this way, the user would develop a feeling for how 
sc notice the submodels are to changes in the various input variables. 

’f I?? would be bt tier able to evaluate predictions made by the entire 
miali ! : ■ -lo jude'e the significance of various changes in model outputs, 
Aiedke# possible value* of using the submodels Individually or In 
at combinations relates to the profile tit not of evaluating 
b‘f e.t |.i Mile I i due aliened assistance ' programs 0 Access to the 

%,** if**- s^wed* I* «e%ht be made available tee local project planners who 
* * n- #fw< *a wifb possible ideas for title I proposals 0 This 

i» e v f . o«f *;•< if soft* If fm at pt eject plaont rs could compare 
* ki * i**#a*m* a ?fb lb? pf e?ec Moos made by ibe submodels and 
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CHAPTER VI 



ORGANIZATION OF THE COMPUTER SIMULATION 



Introduction 

The simulation will be used by the Office of Education to 
evaluate programs in particular communities. It will not be used 
explicitly* to allocate all Title I money in an optimal way; that is, 
the model will not tell us to spend $300, 000 in district A, $20, 000 
in district B, and so on for all of Title X, Rather it will tell us 
that for community A, program 1 yields better results for less money 
than program 2, The simulation will be an evaluation and planning 
tool* not a research instrument. 

The model will compare the effects and costs of proposed programs 
or combinations of programs against a set of Basic effects and costs 
which are derived By projecting current school district operations with 
no changes. In computer terms, this means we will compare alter- 
native program runs against a Base line run, 

Although research use is net the goal of our model development 
effort# it will fee necessary for the user to have the opportunity to 
Investigate in depth the predicted results of different programs as well 
fg the working of the model Itself, An option for in-depth investigation 
enables the user to develop his confidence in and facility with the model 
to whatever level of detail he desires. The user may have a sufestan- 
tive interest in a part of the model such that he wishes to observe 
chillies In variables which are not summarised In the ordinary evaluation 
summary. He will be able t© Investigate these changes. 

The simulation is designed to accomplish four basic tasks? 

11 Preprocessing of the field ^uestiomiatres, 
li Priparaiion of a data base. 

If ©peril ten of the model for baseline and prediction, and 
If lemma #• g of fosi^effectiveness for alternative programs® 
fg* simulation simplifies the Program Administrator*# problem by 
#s«*##Pat*g aft loetfol feactions and automatically handling these different 

f.n.ifu* §es fig**## $ * 



We shall describe the features built into the simulation for 
routine evaluation and why they are needed. We shall follow that 
discussion by describing some of the special features that are provided 
for research investigations. Let us start by stating the basic purpose 
for the simulation, namely, to provide output for each Title I project 
descriptive of the impact that the project is likely to have. 

A prerequisite for operations is the preparation of a community 
data base describing the school or district being simulated. This 
data base is stored on magnetic tape and serves as the focal point 
for information to the cost effectiveness model. In effect, the data 
baso represents the information about the school, Title I, and the 
disadvantaged student. In general terms, a projection is first made 
by the model without the Title I program? that is, a projection 
is made of the expected future effects on the community and students 
with no Title I changes? and, then, a projection is made of the expected 
future effects on the community and students with the proposed Title I 
program added. The difference between these two projections rep- 
resents the Impact of the proposed Title I project. The Program 
Administrator compares these impacts and their associated costs and 
decides which program Is better for the community, c 



The Pre- Processor 

Preprocessing Is Insurance, Pre-processing helps assure that 

it 

the data used hy the model will be self-consistent, that is, that 
mistakes will not be Introduced by erroneous card punching or data 
omission. 

The pre-processing computer program analyses the data from 
the key punched field questionnaires and reports an error whenever 
selected built-in checks for consistency are violated. One such check 
concerns costs. If the sum of the component costs is not equal to 
the whole, an error has been made. Other checks are made using 
lime, characteriitlcif achievement patterns and so forth, A report 



OECE COST-EFFECTIVENESS SIMULATION 
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repeated until the Data Deck results In an error-free 



IS generated by the pre-processor detailing its findings and the 
Program Administrator is asked to make corrections and provide 
the missing data. When the Administrator is satisfied that the 
input is correct, he can instruct the simulation to proceed. 

Data Base Makeup (Figure 3) 

The data base is the repository of information from the 
questionnaires for use by the model. The simulation brings 
together consistent data from the Pre-processor about the school, 
the students, and the Title I project, a set of model coefficients 
chosen from the simulation master file of information, and control 
information necessary to operate the model. The data base is 
more than just a storehouse of information; it is an organized 
arrangement of data in a form ready for immediate use by the 
model. For example, some questionnaire information is keypunched 
as a yes or no answer. The model uses this answer as a variable 
setting. The question is transformed by the simulation in making 
up the data base into the particular variable setting required by the 
model. Let us consider an alternative way of doing this. Suppose 
that the Administrator were required to take the answers on the 
questionnaires and transform them into numbers representing the 
setting of variables. This would then require two things. First, he 
would have to know all the model variables, and second, he would 
have to know how to scale all the settings for each variable properly. 
This would be a great imposition when the simulation can perform 

this task so much more easily directly from the questionnaire 
information. 

. In order to save the Administrator the trouble of making variable 
settings, we provide a data base makeup program as a part of the 
simulation. Let us define a magnetic tape as a master file. Stored 
on this file are all the necessary elements for transforming the 
questionnaire information into the proper form. The master file 
also contains the coefficients needed by the model which remain 
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invariant with different Title I proposals and communities. These 
coefficients embody the most recent research findings and expert 
opinions and, while subject to change, are in the short run, 
invariant. 

The simulation makes up a data base for each community 
project by transforming the questionnaire data according to the 
rules provided, and then combining it with the constant information 
taken from the master file. 

Model Supervisor (Figure 4a, 4b) 

Perhaps the hardest function of the simulation to explain is 
model supervision. The supervisor acts like a big rotary switch, 
performing a sequence of control tasks one after the other. Let us 
look at what happens first, we have to hook up the data base 
without the Title I project, operate the model, and store the 
predictions on our history file. This baseline run provides the 
reference line for measuring the impacts of different Title I programs. 
Next, we have to repeat the process, except that this time we add 
the first selected sub-project from the data base. (The sub-project 
describes a particular target group within a particular project, 
e.g. , third grade remedial reading). Again, we operate the model 
and store the predictions on our history file. We repeat this 
process as long as there are sub-projects to process, building up a 
sequence of results on the history tape. Finally, when there are 
no more sub-projects left for this community the supervisor turns 
its attention to retrieving the stored impacts, organizing them, 
and printing comparative results side by side. The^ supervisor 
keeps these activities separate, in proper sequence, and checks 
that the files are in working order. 

Special Features 

Having considered the operational functions of the simulation, 
let us examine some of the considerations that went into the simulation 
design. The simulation embodies flexibility, special file construction, 
ease of operation, and so forth, but the most important design concept 
in the simulation is modularity. What do we mean by modularity? 
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Think Of a pre-fab house. Each piece comes complete, and the 
pieces can be put together in many interesting ways. The 
simulation is constructed just like the pre-fab house. But why 

go to all that trouble? The main thing we wanted was an opera- 
tional facility and, at the same time, a facility that would allow 
experimentation with different parts of the model. Without modular 
construction we could not gain access to the pieces. Without 
a modular design of the data base we would not be able to operate 
different parts of the model with varying assumptions. 

One way of providing modular construction and flexibility in 
operation is to separate the different simulation functions, such as 
preprocessing, data base makeup, or model operation by intermedia!* 
information files. This creates an added burden of file maintenance 

and updating, but provides and easy way to use the model for 
research. 

One research objective might be independent operation of the 
Instructional Process Submodel. A specialized data base and 
parameter control are available to the user, when the simulation 
is run in this mode. The simulation has been designed to balance 

ease of operation for cost-effectiveness evaluations against alternative 
uses as a research tool. 

Special programs are provided to update the master information 
file. Each part of the model is built in the form of a replaceable 
module. This means that it is as easy for a programmer to replace 
an equation or a coefficient as it is for an electronic technician 
to replace a worn out tube or transistor. 

# 

Summary 

In summary, the simulation controls the processing and error 
checking of the questionnaire data, as well as the operation of 
the Cost-Effectiveness model. It is constructed in a modular way 
that provides a high degree of flexibility in use and future growth, 

yet does not detract from its use as a facility for making routine 
evaluations of Title I fund applications. 
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OUTPUT SUPERVISOR 



Figure 4b 
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